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1. Introduction 

H. 

P^ ■ In this paper we investigate a new family of multivariable polynomials. These polynomials, 

S^ • denoted Rx{zi, . . . ,Zn;r), depend on a parameter r and are indexed by a partition A of 

length n. Up to a scalar, Rx is characterized by the following elementary properties: 
• Rx is symmetric in the odd variables ^1,^3,25,... as well as in the even variables 
^ ■ Z2,Z4,zq, . . .. Polynomials having this kind of symmetry are called semisymmetric. 

Q ' • For the partition A = (Ai, . . . , An) define the odd degree as |A|odd := J2i odd -^*- Then 

^ ! the degree of Rx{z) is |A|odd- 

O , • Consider the vector q := ((n — l)r, (n — 2)r, . . . , r, 0). Then -Ra(^) vanishes at all points 

0\ ■ of the form z = g + fx where fi is any partition with /U 7^ A and |/u|odd < |A|odd- 

■^ ' The simplest nontrivial example comes from the partition (1) = (1, 0, . . . , 0) in which case 

'j^ I Rx{z;r) = J27=ii~^y~^ ^i ~ L^^/4J. It is clearly semisymmetric, has degree |A|odd — 1 

p . and vanishes at z = g + /j where /U = (0) or ^u = (1^). 

<^ • The polynomials -Ra(^) are analogous to the polynomials Px{z;r) which were pre- 

/S ' viously introduced in[KSl]. In fact, the definition of Pa is the same except that Pa is 

cz \ symmetric in all variables zi, . . . , 2„ and the odd degree |A|odd is replaced by the (full) 

degree |A| = ^j A^. The Pa are called shifted Jack polynomials since their highest degree 
components are the Jack polynomials. This is in contrast to their semisymmetric counter- 
parts: even their highest degree components form a genuinely new class of multivariable 
homogeneous polynomials. 

All the polynomials mentioned above have a representation theoretic origin. Let G be a 
connected reductive group acting on a finite dimensional vector space V. We are interested 
in the case when this action is multiplicity free, i.e., every simple G-module occurs at most 
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once in the algebra of polynomial functions of V. Then the algebra PD*^ of G-invariant 
differential operators on V is commutative. Moreover, one can define a (Harish Chandra) 
isomorphism which identifies PD with the space C[a*]^^ of VFv' -invariant polynomials 
on a finite dimensional vector space a* where Wy is a finite reflection group. 

The point is now that there exist very particular invariant differential operators Dx on 
V which form a basis of PD*^. The idea of their construction goes back to Capelli. The op- 
erators Dx correspond, via the Harish Chandra isomorphism, to polynomials pa G C[a*]^^ 
and it is these polynomials which one would like to understand. Already the top homo- 
geneous component px of px is very important since it has the following representation 
theoretic meaning. Consider the symbol Ex of the differential operator Dx- By construc- 
tion, this is a G-invariant function on the cotangent bundle Ty = V (B V* . It can be 
considered as a generalization of a zonal spherical function. On the other hand, one can 
define a (Chevalley) isomorphism of C[y ©y*]*-' with C[a*]^^ and under this isomorphism 
Ex corresponds to px- 

The investigation of the polynomials px and px is greatly facilitated by the fact that 
multiplicity free actions on vector spaces are classified. This is due to the efforts of Kac 
[Kac], Benson- Ratcliff [BR], and Leahy [Le]. The most important numerical invariant of 
a multiplicity free action is the dimension of a* which is called its rank. It follows from 
the classification that there are only seven series in which the rank is unbounded. These 
series are listed in the following table. More precisely, an (indecomposable) multiplicity 
free action which is not in the table has rank less or equal 7. 
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As indicated in the table, the seven series fall into three classes: classical, semiclassical, 
and quasiclassical. The reason for that is that all the cases in each class can be treated 



uniformly: there are polynomials depending on a free parameter* r such that the polyno- 
mials px of each particular case are obtained by specializing the parameter as indicated in 
the table. 

In the classical class, the space V consists of matrices: symmetric, rectangular, or 
skewsymmetric. This case has been treated in [KSl] and the polynomials p\ are basically 
the shifted Jack polynomials Px{z;r). 

The purpose of the present paper is to study the semiclassical case. Here an element 
of y is a pair (A, v) where A is a rectangular or skewsymmetric matrix and z; is a vector. 
The polynomials p\ are our R\{z; r) described in the beginning. 

The quasiclassical class is left mostly to future research. Here an element of y is a 
pair {A, a) where A is a matrix as above and a is a covector (linear form). Preliminary 
investigations indicate that this case is much more involved than the other two cases. 
Nevertheless, the small cases, more precisely the cases with rank < 4, are covered also in the 
present paper since for those the combinatorics of the quasiclassical and the semiclassical 
class turn out to be isomorphic. In particular, we can also say something about the action 
of GLi{C) X GLq{C) and C^ © (C^)*, a case already considered by Vilenkin-Sapiro [VS]. 

The zonal spherical functions E\ have numerous different descriptions. This means 
that the results of this paper are also relevant for the action of GLp_ i(C) on GLp{C) by 
conjugation or for the action of Sp2p{C) on X := SL2p+i{C)/Sp2p{C). This is remarkable 
since the space X is only spherical and not symmetric. 

Now we describe the our results about the polynomials Rx. The most important result 
is the construction of n commuting difference operators of which the Rx are simultaneous 
eigenf unctions. These differential operators are defined by explicit formulas (4.4). An 
analogous result has already been the main statement of [KSl]. The result here is similar 
but much more involved. 

Except for some elementary results, like existence and uniqueness of the Rx, most 
proofs hinge on the difference operators. The first immediate consequence is that the 
top homogeneous component Rx of Rx is a simultaneous eigenfunction of n commuting 

differential operators of order 1, 1, 2, 2, 3, 3, These are semisymmetric analogues of the 

Sekiguchi-Debiard operators which characterize Jack polynomials. Observe that Heckman 
and Opdam define analogues of the Sekiguchi-Debiard operators for any finite root system 
but our semisymmetric case is not covered by their construction. 

Another rather immediate consequence of the difference operators is the Extra Van- 
ishing Theorem. Remember, that Rx is defined to vanish at all points of the form z — g + n 



In the odd rank quasiclassical case there are two parameters. 



where /U 7^ A and |/u|odd < |A|odd- It turns out, that R\ actuaUy vanishes at many more 
points. In section 4 we define an order relation A ^ ^ on the set of partitions such that 
R\{g + n) = whenever A ^ /U. This order relation should be regarded as a semisymmetric 
analogue of the familiar containment relation for partitions. 

A property which can be considered as dual to extra vanishing is called triangularity. 
By definition, the polynomial Rx can be expressed as a linear combination of monomials 
z^ whose degree is less or equal |A|odd- As it turns out much fewer monomials are needed. 
This phenomenon is called triangularity since it can be rephrased as saying that the base 
change matrix from monomials to -Ra's is triangular. In section 6 we actually prove two 
versions of triangularity. For the first, we define a map A 1-^ [A] from the set of partitions 
into the set N" of compositions such that if z'^ appears in Rx then // < [A]. Here "<" is 
the usual (inhomogeneous) dominance order on N^. 

The monomials z^ are not semisymmetric. Therefore, one can attempt to formulate 
triangularity strictly within the set of semisymmetric polynomials. To do this, define the 
elementary semisymmetric polynomials as ei := -R(i), ^2 '■= -R(i2), etc. These can be 
computed explicitly: 62^-1 = e^^^ — ef^*^" and 62^ = ef^*^" where e° /<^^<^" jg ^j^g usual 
elementary symmetric function of degree i in the variables {z2j-i | j } or {z2j \ j }, 
respectively. Now consider all monomials in the e^: 

(1.1) e^ := ef -'^^ef -'^^ . . . e;;ri'"''"e^" 

where /i is a partition. In section 6 we define a new order relation |U ^ A on the set 
of partitions which is a semisymmetric analogue of the classical dominance order. Using 
the explicit form of the difference operators we are able to prove that each Rx is a linear 
combinations of e^ with n ^ X. This is the second triangularity result, alluded to above. 
It should be noted that the second form easily implies the first one but not conversely. 
This has to be seen in contrast to the classical case of (shifted) Jack polynomials where 
both forms of triangularity are actually equivalent. 

In section 7, we prove what could be considered as the second main result of this 
paper: the duality formula (7.6) 

^■^ Rxi-a-g) ^^ ' R^.{Q + li)R^.{-a-Qy 

where a is an arbitrary parameter and a = («,...,«). In other words, the formula 
expresses the transformation Zi 1-^ —a — Zi of the space of semisymmetric polynomials in 
terms of its basis Rx- The classical analogue has been established Okounkov [Ok] whose 



proof we follow closely. This holds also for some of its consequences described below. The 
key to the proof of the duality formula are again the difference operators. 

A first consequence is an explicit interpolation formula (Theorem 7.6iii)). It allows 
to explicitly calculate the expansion of an arbitrary semisymmetric polynomial in terms of 
R\s. More precisely, 

(1.3) /„).5:,-i)i.u-i^ «,.(.) 

where 

(1.4) fig + ^) = ^(-i)IHo.. -R^(g + ^) ;(^ + ^). 

^ R^{g + u) 

The key to this formula is the observation that the transformation z ^-^ —a — z is involutory. 
Another consequence of the duality theorem is the fact that the expression 

(1.5) Rxi-a-g-iy) 

R\{-QL- g) 

is symmetric in A and z/ (just put z = g + u in (1.2)). This observation allows sometimes 
to interchange in formulas the argument z with the index A. For example, let D be one 
of the fundamental difference operators. Then the eigenvalue equation D{Rx) = c{X)Rx 
turns into a Fieri type formula, i.e., a formula which expresses the multiplication operator 
p ^-^ fp on the space of semisymmetric polynomials (with fixed /) in terms of the basis 
Rx. This way, we obtain in section 9 the expansion of f{z)R\{z) in terms of -R^'s where 
f{z) is one of e°^'^(2), e1^'^^{z), or Rii{z). As a byproduct of these investigation we prove 
in section 8 a formula for the value of Rx{z) in z = —a — g. 

Most of these results have consequences for the homogeneous polynomials Rx- The 
evaluation formula specializes to a formula for the value of Rx{z) in z = (1, . . . , 1). The 
duality formula implies the semisymmetric binomial theorem which expresses the effect of 
the transformation Zi ^^ a + Zi in terms of the homogeneous basis Rx (see section 8 for 
these two statements). Its classical analogue is due to Okounkov-Olshanski [00]. Finally, 
we obtain expansions of f{z)Rx in terms of -R^^'s where f{z) is one of e1'^'^{z), e|^^"(2), or 

Finally, scattered all over the paper, we derive several explicit formulas. More pre- 
cisely, we determine Rx{z) when A = (al"^~^), a,m > 1 is "hook" (Corollary 2.8 for 
a = 1, Corollary 4.10 for m odd. Corollary 9.5 for m even). Furthermore, we calculate 
Rx{z) where A = (a 6) is a two row diagram (Theorem 5.4) or any Rx if n = 3. For 
n = 3 these are expressible in terms of Jacobi polynomials, for n = 4 we get one of Horn's 
hypergeometric function. 



Even though most of the theory is parallel to that of (shifted) Jack polynomials there 
are some differences. One of them is that the Rx don't specialize for r = to anything 
easy. It seems that matters become rather more involved. Also, neither R\ nor its top 
homogeneous term Rx seems to have any obvious positivity properties (see [KSl] and 
[KS2] for the classical case). Another remarkable difference occurs when the number of 
variables is even. Then the specialization of Rx{z) at the point z = (!,...,!) may be zero. 
This has the consequence that the shifted polynomials R\{z) cannot be defined via the 
binomial formula (8.17) since not all of them occur in this formula. A major open problem 
is orthogonality: Jack polynomials are most commonly defined by an orthogonalization 
process with respect to some explicit scalar product. Such a scalar product is still missing 
for the -Ra's. 

Acknowledgment: I would like to thank Yasmine Sanderson and the referee for valuable 
comments concerning the exposition of this paper. 

2. Shifted semisymmetric functions 

Let /c be a field of characteristic zero. Consider the polynomial ring V := k[zi, . . . .z^]. 
On it, the symmetric group Sn acts by permutation of the variables. The semisymmetric 
group is the subgroup W of Sn which doesn't mix even and with odd entries: tt G VF if 
nil) = imod2 for all i. Throughout this paper, we are adopting the following notation: 

we put n := [n/2j and n := n — n = \n/2] . Then we have W = S^ x Sn- For ^ G /c"" we 

let ^odd := (^1, ^3, • • • , Z2n-l) e k"^ and Zeven ■■= (^2, ^4, • • • , Z2n) € k-. 

We are going to study the ring of semisymmetric polynomials V^ . Clearly, as an 
algebra, V^ is a polynomial ring generated by ei{zodd), i = l,...,n and ei{zeven), i = 
1, . . . , n where e^ is the i-th elementary symmetric polynomial. 

Let A be the set of partitions of length n, i.e., n-tuples of integers A = (A^) with 
Ai > . . . > An > 0. We are going to consider A as a subset of k"^. The degree of 
A is |A| = J^i'^i- We also define the odd degree |A|odd := |Aodd| and the even degree 
|A|even := |Aeven|- The odd degree will be for semisymmetric polynomials what the degree 
is for symmetric polynomials. 

Finally, we choose once and for all a parameter r ^ k with* r ^ Q<o and put 

(2.1) g := {{n - l)r, {n - 2)r, . . . , 2r, r, 0). 



* In fact, only the slightly weaker condition r 7^ — ^ where p and q are integers with 
1 < p and 1 < q < ^ is needed. 
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2.1. Theorem. For any d E 'N let A{d) be the set of X E A with |A|odd < d. Let 
A{d) ^' k : X \-^ a\ be any map. Then there is a unique f E V^ with deg/ < d and 
f{g + A) = ax for all X E A{d). 

Proof: We are using induction on d + n. To make the dependence on the dimension n 
explicit we write it as an index. We have A^i-i ^-^ A^ by appending a zero. Moreover let 
V^"i^ -^ V^" : (7 h^ (7+ be the homomorphism which maps ei{Zodd / even) ^ V^^^^ to 
eiizodd / even) ^ ^tT" • Then deg5r+ = deg^r and g+{zi, . . . ,z^-i,0) = g{zi, . . . ,z^-i). 

Let e{z) := Yli n-i even^i- Observe dege = n. Since e{z) is the one generator of 
V^" which is not in the image of V^^{^ , every / E V^" can be uniquely expressed as 
f{z) = g+{z)+e{z)h{z) with g E V^^{\ degg < deg/, h E V^" , and deg/i < deg / - n. 

Now we split An{d) into two parts An{d)^ and An{d)^ according to whether the last 
component A^ is zero or not. 

For any g E Vn-i let go{z) := g{zi +r, . . . , z^-i + r). Clearly, we can identify An{d)^ 
with A„,-i{d). Then for any A E An{d)^ we have 

(2.2) g+iQr, + A) = ^(Ai + {n- l)r, . . . , A^-i + r) = go{gn-i + A). 

Since e(^ + A) = for every A with A^ = the system of linear equations f{Qn + X) = 
ttA, A E An{d)^ is equivalent to the system (7o(^n-i + A) = a\, X E An-i{d). By induction 
on the number of variables we conclude that is has a unique solution. 

For any A E An{d)^ holds e{g + A) 7^ since, by assumption, r ^ Q<o- Thus, we can 
define a^ := {ax — g~^{g + X))/e{g + A). The map A h^ A := (Ai — 1, . . . , A^ — 1) identifies 
An{d)^ with An{d — n). Thus the system of linear equations f{g + X) = a\, X E An{d)^ is 
equivalent to the system h{g + A) = a^, A E A„((i — n) where h{z) = h{zi — 1, . . . ,Zn — I). 
By induction on the degree we conclude that is has a unique solution, as well. D 

Now, we can define interpolation polynomials as follows: 

Definition: For every A G A let r>(z; r) be the unique polynomial such that 

• it is VF-invariant, 

• its degree is d := |A|odd, 

• for all fx E A with |/u|odd < d holds rx{g + n;r) = dx^i (Kronecker delta). 

The normalization rx{g + X;r) = 1 is very natural but there is one which is often more 
convenient: the "leading" coefficient should equal to one. To define what that means, 
observe that every VF-orbit of a monomial contains exactly one monomial, say z'^ , such 
that both z/odd and z/even are partitions. These u are in bijection with A. In fact, for every 



partition A G A we define the composition [A] G N" by 

(2.3) [A]m := Xm — Xm+i + • • • + (—1) A^. 

Since [A]^, = (Am, — Xm+i) + [Mm+2j both [A]odd and [Ajeven are in fact partitions. Con- 
versely, let z/ be a composition such that both z/odd and Ueven are partitions. Then 

(2.4) A = (z/i + Z/2, 1^2 + «^3, ^3 + ^^4, «^4 + «^5, • • •) 

is in A. One easily checks that these two maps are inverse to each other. Of special interest 
is the first component of [A] since 

(2.5) [A]i = |A|odd — |A|even = Ai — A2 + A3 \- ■ ■ ■ 

In particular, 

(2.6) [A]i =0 if and only if Ai = A2, A3 = A4, . . . , and An = in case n is odd. 

Moreover, we have |[A]| = |A|odd and therefore degrx{z;r) = degz^'^^. 

2.2. Proposition. The coefficient C \{r) o/z^'^l inr\{z]r) is non-zero. 
With this result we can define the renormalized polynomial 

(2.7) Rx{z; r) := ^^rx{z; r) = z^^^ + ... 

We are proving the proposition by computing Cx{r) explicitly. For this we need some more 
notation. A partition A can be represented by its diagram, i.e., the set of all {i,j) G N^ 
(called boxes) such that 1 < i < n and I < j < Xi- The dual partition A' is defined by the 
transposed diagram {(j, «) | (^,j) G A}. For every box s = {i,j) G A we define the arm- 
length ax{s) := Xi — j and the leg-length l\{s) := A' — i. Then we define 

(2.8) [cl(r)]even := Hiaxis) + l + lxis)r). 

sg A 
I -^{s) even 

For example, we have [C(^)(r)]even = a!, [C(^^)(r)]even = (a - 6)!6!, and [c'^^m){r)]even = 
Y[i<i<m (1 + ir). 

i even 

2.3. Lemma. For every A G A holds C\{r) = [c'xir)]~^^^. In particular, we have 

(2.9) i?A(^ + A;r) = [c';,(r)]even. 

Proof: We retain the notation of the proof of Theorem 2.1 and prove the lemma by a 
similar induction. In particular, we have an expression r\{z) = g~^{z) + e{z)h{z). 



If An = then e{g + A) = and therefore g{z) = rx/{zi — r, . . . , z^-i — r) where the 
prime means "drop the last component". Moreover, the coefficient of z^^l in rx equals the 
one in g (observe [A]^ = 0). Thus, we get by induction Cx{r) = Cx'{r) = [cy {r)]~J^^. But 
we also have [c^(r)]even = [cA'(^)]even which finishes this case. 

If An > 1 then g{z) = and h{z) = e{g + X)~^r^{zi — 1, . . . , ^n — !)• One checks z^^l = 
e{z)z^'^^. Thus, by induction, the coefficient of z^^'^ is e{g + A)~^[c~(r)]^gjj. But e{g + A) 
is the contribution of the first column of A to [c^(?')]even- Thus we get Cx{r) = [c'x{r)]~J^^^, 
as claimed. D 

The second case of the preceding proof gives the following recursion formula which allows 
to reduce the computation of Rx to the case A^ = 0. 

2.4. Corollary. Let 5 := (1, . . . , 1). Then for every A G A with A^ > 1 holds 

(2.10) Rx{z;r) = { U z^ ) ■ Rx-s{z - 5;r). 

n—i even 

We also have the following stability result: 

2.5. Proposition. For z = [zi, . . . ,Zn) G k'^ let z' := {zi, . . . , z^-i) G k'^~^ . Then we 
have for any A G A.- 

(2.11) i?.(.i,...,.n-l,0) = {^^'(^l-^'---'^-l-^) ^/>=.0' 

^- U otherwise. 

Proof: If An > 1 then Rx is divisible by z^ (Corollary 2.4), hence Rx\zn=Q — 0. Otherwise, 
Rx\z„=o satisfies the definition of Rx'{zi — r, . . . , Zn-i — r). D 

Remark: In many circumstances it is more convenient to consider the polynomials 
Rx{u;r) := Rx{g + u;r). Their main advantage is that the stability result above can 
now be expressed as 

(2.12) Rx{Ui,..., Wn-l, 0) = Rx'{Ui,..., -Un-l) 

whenever An = 0. This means that one can form a theory of shifted semisymmetric polyno- 
mials which is independent of the dimension n. For this one defines them in infinitely many 
variables as follows. Let Poo be the projective limit of the polynomial rings /c[wi, . . . , Un] in 
the category of filtered algebras. An element of Poo is a possibly infinite linear combination 
of monomials in ui,U2, ■ ■ ■ whose degrees are uniformly bounded. Let Aoo be the set of all 
descending sequences of integers Ai > A2 > . . . with An = for n » 0. The stability result 
above says that for any A the sequence (-R(Ai,...,a„)('U'1, • • • ,'U'n))n>o is an element of Poo- 
It is denoted by Rx- 
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The drawback of this method that the action of the semisymnietric group gets dis- 
torted. More precisely, W acts now hj tt • u := Tr{u + g) — Q- For example, the simple 
reflection Sii^2 acts as Ui t— > Ui^2 — 2r and Ui^2 ^^ w^ + 2r. This action extends to an 
action of Woo on Voo where Woo is the group of parity preserving permutations of N with 
finite support. It is easy to show that the R\, X G Aoo form a linear basis of V^°° . 

Next, we present some compatibility results with shifted Jack polynomials. First, we 
recall their definition from [KSl]. More or less, one has to replace the semisymmetric 
group by the full symmetric group and the odd degree by the full degree. More precisely: 
for each A G A we define Px{zi, . . . jZn'jr) as the unique polynomial having the following 
properties: 

• P\ is invariant under the full symmetric group S^; 

• degPx = |A|; 

• the coefficient of ^'^ is 1; 

• P\{g + /u) = for all /U G A with \fx\ < |A| and /U 7^ A. 
Analogously, we define P\{u; r) := P\{g + u;r). 

Now we show that the symmetric polynomials P\{z;r) are in two ways special cases 
of the semisymmetric polynomials R\{z; r). 

2.6. Theorem. Let X e A with [A]i = (see (2.6);. Then 

(2.13) ^A(t^;r)=PA_n(«even;2r). 

Proof: We show that the polynomial P on the right hand side matches the definition of 
Rx. First, observe that the shifted action of W induces on the even coordinates the shifted 
action of Weven with parameter 2r. Thus, P is shifted semisymmetric. Moreover, we have 

(2.14) |A|odd = |Aodd| = |Aeven| 

which shows that the degree of P is correct. Since [A] = (0, A2, 0, A4, . . .) we have z'-^^ — 
^even^ . This shows that the normalization of P is correct, as well. 

It remains to check the vanishing conditions. For this let /U G A with |/u|odd < |A|odd = 
|Aeven|- Then 

(2.15) l/^evenl ^ |/^odd| ^ |Aeven|- 

This implies -P(/U) = unless fXeven = Aeven- But then < [/U]i = |/U|odd — |A*even| < 

|A|odd — |Aeven| = which implies fxi = /U2, /U3 = /U4, etc., i.e., fx = X. D 
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The other connection between R\ and Px is: 

2.7. Theorem. For every n = (/Ui, . . . , fin) G A^ holds 
^2 lQ^ sr^ R\{u;r) ^ P^{uoM;2r) 

Proof: Let P be the polynomial on the right hand side of (2.16). Again, the shifted action 
of W induces on the odd coordinates the shifted action of Wodd with parameter 2r. Thus, 
P is shifted semisymmetric. In particular, we have an expression 

(2.17) P = Y1 ^^^^' ^'^^ l^l°dd < H 

A 

Suppose there is A with c\ j^ and Aodd 7^ A*- If we choose one of minimal degree, the 
left hand side of (2.17) evaluates at w = A to ca while P(Aodd) = 0. Contradiction. Thus 
c\ = unless Aodd = A*- ^^ that case, the value of cx is immediately obtained by evaluating 
both sides of (2.17) at -u = A. D 

As a corollary we get a formula for the elementary semisymmetric polynomials: 

2.8. Corollary. 

(2.18) R(^i2m~i-){u;r) = P(ir„)(wodd; 2r) - P(ir„)(weven; 2r) 

(2.19) R(^i2m-){u; r) = P(irrx)(weven; 2r) 

Proof: Formula (2.19) is a special case of (2.13). If we put A = (i^^^-i^ j^ (2.16) and use 

(2.19) we get 

(2.20) R(^i2m-i^{u;r) = Q;P(i«x)(wodd; 2r) - /3P(ir„)(weven; 2r) 

with two constants a and /3. Comparison of the coefficient of u^^^^ ^^ = uius . . .U2m-i 
implies a = 1. Next we evaluate (2.20) at -u = (1^"^). The left-hand side is zero by 
definition. Then (l2'^)odd = (!"") = (l^"')even implies /3 = 1. D 

Explicit formulas for P(^im^[u] r) can be found in e.g. [KSl] 3.1. One them is: 

m 

(2.21) P(i^)(t.;r)= J^ JJ(^^^. + (_^- _ 1)^) 

n>ii>i2>...>im>l i=l 
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Thus, the first few elementary semisymmetric polynomials are 

(2.22) R^i){u;r) = ei(wodd) -ei(weven) = {ui + U3 + . . .) - {u2 + U4 + ...)., 

(2.23) ^(11) (-U; r) = ei (Weven) = W2 + ^4 + • • • , 

(2.24) R{iii)iu;r) = e2{uodd)-e2iueven)+r '^{i-l)ui-r "^ {i - 2)ui, 



I o 



dd 



(2.25) i?(iiii)(w;r) = e2('Ueven) + r ^ (z - 2)^^. 



I even 



Let R\{z;r) be the top homogeneous component of R\{z;r). Since the highest degree 
component of P(im) is the elementary symmetric function e™, we obtain: 

2.9. Corollary. Let R\{z;r) be the highest degree component of R\{z;r). Then 

(2.26) R(^i2m-l){z;r) = CmiZodd) - em{Zeven) 

(2.27) R(^i2m^{z;r) = e^(2even) 

We conclude this section with a list of all polynomials R\ which are non-elementary 
of degree at most 3, i.e., with |A|odd < 3 and Ai > 1. Each Rx is expressed as a polynomial 
in the Rni). This means, that the formulas are valid for all n with the convention that 
Rx = if the length of A is greater than n. 

R{2) = Rfi) - ^(1) 
R{2i) = R{i)R{ii) - 1^^(111) 
R(22) = Rf^^^ - j^R(iiii) - R(ii) 
R{2ii) = R{i)R{iii) — R(iii) 
-R(2iii) = -R(i)-R(iiii) — i:p47-R(iiiii) 

R(221) = R{11)R{111) — T+2f -^(l)-^(llll) ~ 1+27-^(11111) — -R(lll) 

(2.28) -R(22ii) = -R(ii)-R(iiii) - 4;rp]--R(iiiiii) - 2-R(iiii) 

R{3) = Rfi) - 3-R^i) + 2-R(i) 

R{3i) = R{i)R{ii) " T+:7-R(i)-R(iii) - -R(i)-R(ii) + i+7-R(iii) 

^(32) = ^(1)^^11) - T^R{ll)Rilll) - i^R{l)R{llll) + (i+^)fi+2r)^(lllll) 

- i?(i)i?(ii) + Y^i?(iii) 

^(33) = Rfii) - 1+7^(11)^(1111) + (i+^)(i+2r)-R(llllll) - ^Rfll) + 1+7-^(1111) 
+ 2i?(ii) 

These formulas were obtained with the help of a computer. 
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3. Representation theoretic interpretation 

Before we study the polynomials R\{z; r) further, we describe the representation theoretic 
interpretation of the three special cases which are mentioned in the introduction. For this, 
we recall some basic facts about multiplicity free representations. Details appeared for 
example in [Knl]. 

Let G be a connected complex reductive group. A finite dimensional representation V 
of G is called multiplicity free if every simple G-module appears in P := C[V] at most once. 
Equivalent to this condition is that a Borel subgroup of G has a dense orbit in V. Thus, 
as a G-module, we have a decomposition P = ©aeAv^Pa where Pa is the simple module 
with lowest weight —A. Then Ay is a set of dominant weights which can be shown to be a 
free abelian monoid (i.e., isomorphic to W). Clearly, all non-zero polynomials in P^ have 
the same degree, denoted by |A|. 

The symmetric algebra D := S*{V) then decomposes accordingly as D = ©AeA^^I^A 
where Da is isomorphic to P^. In particular, A is its highest weight. The space D can 
be interpreted either as polynomial functions on V* or as constant coefficient differential 
operators on V. Accordingly, P(g>D can be identified with either the algebra of polynomial 
functions on 1/ ©]/* or the algebra PD of linear differential operators on V with polynomial 
coefficients. 

The point is now that the space of G-invariants (P®ID))*^ comes with a (up to scalars) 
distinguished basis: we have 

(3.1) (P©D)^=_ ©^ (Pa©D^)^. 



A,/ieA 



V 



Each summand is zero unless A = /U in which case it is one-dimensional (Schur's Lemma). 
We denote a generator as E\ if regarded as a function on F © F* (called a zonal spherical 
function) and D\ if regarded as a differential operator (called a Capelli operator). 

The Capelli operators are easier accessible, whence we start with them. Each differ- 
ential operator D E (PD)'-' acts on Pa by a scalar denoted Ci:)(A). Recall that Ay is a set 
of weights and therefore sits in t*, the dual of the Cartan subalgebra. Let a* be its C- 
span. Let W C GL{i*) be the Weyl group and let ^ G t* be the half-sum of the positive 
roots. Then the shifted action of W on t* is defined hj w • x = ^{x + q) ^ Q- 

3.1. Theorem. ([Knl] 4.4, 4.8, 4.9, 4.7). Let V be a multiplicity free representation, 
a) Each cd is the restriction of a unique polynomial (also denoted cd) on a*. 
h) There is a subgroup Wy C W such that a* C t* is Wy -stable with respect to the shifted 

action and such that D \-^ cd is an algebra isomorphism of (PD)*-' with C[a*]^^, the 

space of shifted Wy -invariant polynomials on a*. 

13 



c) The "little Weyl group" Wy acts as a reflection group on a*. In particular, (PD)*-^ 
and C[a*]^^ are polynomial rings. 

Since (PD)*-' has a distinguished basis we obtain a basis cx = cd^ of C[a*]^^. There is a 
purely combinatorial characterization of the cx: 

3.2. Theorem. ([Knl] 4.10) The polynomial cx G C[a*]^^ is, up to a scalar factor, 
characterized by the vanishing condition ca(/u) = for all n G Ay with \fx\ < \X\ and fx ^ 
A. 

One can eliminate the shifted action of Wy as follows: choose a VF\/-stable complement 
ao of a* in t* and let ^ = ^ + ^o with ^ G a* and po G Oq. The condition that a* is shifted 
VFy-stable means wq — ^ G a* for all w G Wy. Thus, ^o is VFy-fixed. Therefore, we 
can define the shifted VFy-action as well with q replaced by q. Actually, one can add 
to Q any fixed vector in a* without changing the shifted action. The point is now that 
Pa(x) := ca(x — ^) is a truly VFy-invariant polynomial on a*. 

3.3. Corollary. The polynomial px G C[a*]^^ is, up to a scalar factor, characterized by 
the vanishing condition Px{.Q + Ij) =0 for all n G Ay with \n\ < |A| and /U 7^ A. 

Now we say a few words about the zonal spherical functions Ex- One of their main 
features is that they have many different interpretations. First, we can consider V (B V* 
as the cotangent bundle of V. Then the symbol of Dx is Ex- This is our principal method 
for their study. 

It is possible to define Ex without reference to Capelli operators. Every differential 
operator D G PD(y) is also a differential operator on 1/ © y* by acting on the first 
argument. As such it is denoted D^^K Observe, that the eigenspaces of PD are then 
just the spaces Pa® ID). Therefore one can characterize Ex as the (up to scalar) unique 
G-invariant function f onV®V* with D'^^\f) = CD{\)f for all D G PD*^. Clearly, it 
suffices to let D run through a set of generators of PD*-' . 

There is also a "Chevalley isomorphism" for G-invariant functions oivV ®V*: 

3.4. Theorem. ([Knl] 4.2, 4.8, 4.5) There is v* G V* and a linear embedding a* ^^ V 
such that the restriction map f 1-^ f\a* x v* induces an isomorphism (P^D)*^ ^ C[a*]^^. 
Moreover, the image of the symbol of D E (PD)*^ is the highest degree component cd of 
cd- In particular. Ex is mapped to cx- 

The subspace a* is constructed as follows: choose v* G V* in the open G-orbit. Then 
choose a Borel subalgebra b = t© u C LieG such that bv* = V* . This is possible, since 
also V* has a dense orbit for any Borel subgroup. The surjective map b ^^ V* : ^ ^-^ ^v* 
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induces the dual injective map i : V ^^ b* . Via the projection b -^ b/u = t we have 
t* C b*. Now, one can show that l{V) fl t* = a* which furnishes us with the desired 
embedding a* ^^ V. 

Theorem 3.4 indicates another way to interpret Ex. Let H* C G he the isotropy 
subgroup of V*. Then the orbit Gv* is open in V* and isomorphic to G/H*. Therefore, a 
function fonV(BV* is G-invariant if and only if its restriction to V^ x z;* is if * -invariant. 
Thus the restriction Ey*^x{v) := E\{v,v*) is the (up to a scalar) unique H* -invariant 
function fonV with D{f) = CD{^)f for all D E PD*-'. The restriction map from V to a* 
defines now an isomorphism of the algebra of i7*-invariants with C[a*]^^. Thereby, the 
function Ey*^\ is mapped to the highest degree component c\ of cx. 

Another interpretation is as follows: let i^ C G be a maximal compact subgroup. Let 
VuheV regarded as a real vector space. It is equipped with a complex conjugation v \~^ v. 
Then we can regard D as the algebra of polynomials in the antiholomorphic variables Zi and 
P (g) D is the algebra of all C- valued polynomials on V^. Thus, the polynomial v i-^ Ex{v, v) 
is the (up to a scalar) unique K -invariant function f on Vm. with D{f) = CD{^)f for all 

D e PD'^. 

Observe that also V has a dense G-orbit Gv with isotropy group denoted by H. By 
restriction we can interpret Ex also as G-invariant function on G/H x G/H*, as an H*- 
invariant function on G/H, or as a function on G which is constant on double cosets for H 
and H* . In this last form. Ex can be interpreted purely representation theoretically: Let 
M a simple G-module which is isomorphic to Pa for some A. Then M^ and (M*)^ are 
both one dimensional, generated by vectors mx and ax, respectively. Then g \-^ Ex{gv, v*) 
equals (up to a scalar) the matrix coefficient g \-^ ax{gmx)- In fact, if we identify M with 
Vx then m is just the evaluation / i-^ f{v). Similarly, M* = 3x and a is evaluation in v*. 
Finally, Ex corresponds to the canonical pairing M x M* -^ C 

Now we are in the position to explain the representation theoretic relevance of the 
polynomials Rx{z;r). 

The case of G = GLp{C) x GLq{C) acting on V := (Cp^C?) © C?. 

The following data are taken from [Knl] p. 315. Put n := min(2p + 1,2^). Then n = 
min(p-|- 1, q) and n = min(p, q). Let Ei and e[ be the weights of the defining representation 
of GLp{C) and G'Lg(C), respectively. Moreover, let iVi := '^^^i^i, oj[ := X]i=i^i- Then 
Ay is the free abelian monoid generated by Wi-i + uj[ for i = 1, . . . ,n and Ui + uj[ for 
i = 1, . . . , n. Thus, if we put e2i-i := £^ for z = 1, . . . , n and e2i := £i for i = 1, . . . , n 
then Ay consists of all x = XlILi -^i^* where (Ai,...,An) is a partition. The degree 
function is such that \uJi\ = and |a;^| = i which translates into |x| = |A|odd- The 
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little Weyl group consists of all permutations of the Si and e[ separately, i.e., Wy is the 
semisymmetric group. Finally, we have g = (^^,2^,...;^^,^^,...). Thus, if we 
project it to the first n + n coordinates and shift it by a suitable VFy -fixed vector we arrive 
at ^ = (^^, ^^, . . . ; ^^, ^^, . . .) = ^ Z]i(^ ~ 0<2i- This shows r = |. In particular, 
c^{x) is a multiple of Rx{x; ^). 

Now we describe the combinatorics in more classical term. For this, it is convenient 
to write 

(3.2) y = (CP©C)®C« = CP+^®C'?, 

i.e., V is the space of {p + 1) x g-matrices X acted upon by 

(3.3) G = GLp{C) X GLq{C) CG:= GLp+i{C) x GLq{C) 

by X ^ AXB^ with A G GLp{C) C GLp+i{C) and B G GLq{C). 

Let Aoo be the set of infinite partitions, i.e., descending sequences of integers ri > 
T2 > • • • with Ti = for z » 0. The length £(r) is the maximal i with r^ 7^ 0. Every 
r G Aoo with £(r) < p parametrizes an irreducible (polynomial) representation Mr of 
GLpiC). 

Let n := min(p + l,q). Then it is well known (see e.g. [GW] Thm. 5.2.7) that there 
is a decomposition of G-modules: 



(3.4) P= XI M^^^^^^M^'^l 



T-eAoo 



Recall also the branching law of GLp+i{C) to GLp{C) (see e.g. [GW] Thm. 8.1.1): as a 
GLp (C)-module we have 

(3.5) M^P+i) = J2 ^i"^ 

a 

where a runs through all partitions with £(cr) < p and which are "interlaced" with r, i.e., 
with Ti > ai > r2 > (72 > . . .. Thus we have £((t) < n := min(p, q). Now we can make 
the decomposition of P into simple G-modules more explicit. Combine r and a to a single 
partition A by putting X^i-x '■— r^ and X^i :— Oi. Then, as a G-module, we have 



(3.6) P = X Pa with Pa = M^^^ ® m[ 



r(p) ^ A^(9) 
A6A 



Here, we use the fact that i{X) < n := n + n = min(2p + 1, 2q). Therefore, one can regard 
A as an element of A = A^. 
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This gives also a nice interpretations of the comparison theorems 2.6 and 2.7. Let 
y = C^ (g) C^ be the space of p x q matrices. Since V projects onto V we have C[y] C P. 
More precisely, 

(3.7) C[V']= Yl ^>- 

ASA 

Thus, every Capelli operator on V can be regarded as a Capelli operator on V. This is 
reflected in formula (2.13). 

On the other hand each G-invariant Capelli operator on V decomposes as a sum of 
G-invariant Capelli operators on V. This is the origin of formula (2.16). 

We can make this fully explicit for the generators -D(io). Let A E V he a, {p + 1) x q- 
matrix. For subsets / C [p + 1] := {1, . . . , p + 1} and J C [q] := {1, . . . , q} of the same size 
i let 

(3.8) detj{A) = det (aij) iei 

be the corresponding minor. These form a basis of MJ-^^^^ M^^f,. = A'{Cp+^)* A^(C'?)*. 
If V is parametrized by the coordinate functions aij let Oa be the matrix with entries 
Then the classical G-invariant Capelli operators on V are 



(3.9) C,:= J2 det/(A)det/(a^), i = l,...,n 



IC[p + l],JC[q] 
\I\ = \J\=i 



Now each A*(Cp+^)* decomposes as a G-module into two pieces: 

(3.10) A^ {cp+^y = A^(cp © c)* = A'{cpy © a'-\cp)* 

Thus also Ci decomposes as Ci = -D(i2i-) + D(^i2i-ij with 

(3.11) 1^(12.)= Yl det/(A)det/(aA), D(i2«-i) = ^ det/ (A) det/ (9^) 

IC[p],JQlq] -rC[p+l],JC[q] 

\i\ = \j\=i p+iei,\i\ = \J\=i 

For example, for n = 3, i.e. p + 1 = q = 2, we have 

d d 

(3.12) 15(1) =a2i^ ha22- 



da2i da 



22 



2 



(3.13) 15(11)= an- hai2^— 

^ ' oaii oai 

(3.14) 15(111) = (ana22 - ai2a2i)(^— ^- 

oaii oa22 oai2 oa2i 
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Now, we explain the zonal spherical functions. We identify V* with the space of 
qx{p + l)-niatrices and the pairing V xV* -^ C is given by {A, A*) \-^ tr{AA*). By 
definition, Ex is a G-invariant functions on 1/ © y* which is a joint eigenvector of the 
differential operators -0(1*) (see (3.11)) acting on the first factor. To make the Chevalley 
isomorphism from Theorem 3.4 explicit, we define for p,q > 1 the following two matrices 
" -%,q{zi,...,Zn) eV 8indE*^^eV*: 

Z2i iii= j, i <P 

z-ii - Z2i+i iii < j, i <P 
U2j if z = p + 1 



^p,q 



(3.15) (Sp, 



QJT-J 



ifz 
otherwise 

Here, we put Ui := zi — Z2 + zs — \- . . .± Zi and z^ 

For example, for p = 3 < q, n = 7 we have 



and 




ifi=j 

if i = q < j 

otherwise 



p + 1 



(3.16) S3,, 



/Z2 




\u2 



Z2 - Z3 
Z4 



^4 



Z2 - Z3 

Za — ^5 

Zq 



Z2 
Z4 



zz 

Z-, 
Zl 



U7 



...J 



"3,9 



/I 










1 










1 









1 





while for p > q = 3, n = 6 we have 



(3.17) Ep,s{z) 



(Z2 







Z2 - Z-i 
Z4 






Z2 - Zz 
Za -^5 

zq 




-p,3 



10 0- 


■ 


10 0- 


■ 


10- 


• 1 



\U2 U4 Uq J 

Let Bp C GLp{C) be the subgroup of upper triangular matrices. Let Bg C GLq{C) be 
the stabilizer of the fiag (fi), (fi, V2), ■ ■ ■ where Vi := Ci + . . . + Cg and where e^ is the i-th 
canonical basis vector of C^. Then B := Bp x Bg is a Borel subgroup of G. One can verify 
by a straightforward but tedious calculation that z 1-^ Ep^g{z) is the embedding a* ^^ V 
when one follows the recipe described after Theorem 3.4 using v* = S* and the Borel 
subgroup B. 

It follows that for every zonal spherical function Ex{A, A*) the restriction Ex{'E,{z), S*) 
is proportional to Rx{z; ^). Since, Ex is the symbol of Dx we obtain Rx{z] ^) also from 
Dx by replacing all coordinate functions aij by S^j {z) and all derivations -^— by 1 if z = j 



orz=p + l>j = Q and otherwise. For example, in the case p + 1 
according to (3.12)-(3.14): 



q = 2 we get 



(3.18) 15(1) 



W3 



Zi- Z2 + Z2,= R[l){z), 15(11) ^ Z2= R{ll){z) 
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(3.19) 1^(111) ^ Z2U3 - {Z2 - Zs)u2 = Z1Z3 = i?(iii)(2). 

For the other interpretations of zonal spherical functions we just mention the case when 
p + 1 = q since that is the only case when the isotropy groups H and H* are reductive. 
In fact, in that case we have H = H* = GLg_i(C) embedded diagonally into G. Thus, 
the action of H* on V is just the action of GLq_i(C) C GLq{C) by conjugation on q x q- 
matrices. The matrix S* is the identity matrix Ig. Thus, the function E\{A,Iq) is a 
joint eigenfunction of the differential operators -0(1^) which is invariant under conjugation 
by GLq-i{C). Any conjugation invariant function is uniquely determined by its value in 
B,p^g{z) and we have Ex{B,p^q{z),Iq) = R\{z; i). 

The case of G = G'L^(C) acting on V = A^C^ © C" : 

We keep the notation of the previous example. According to the data in [Knl] p. 314, the 
weight monoid Ay is freely generated by cu^ for z = 1, . . . , n. Thus, if we set Ci := Si for 
all i then Ky consists of all x = SiLi -^«^* where (Ai, . . . , A^) is a partition. The degree 
function is given by \uji\ = \^]. Thus |x| = |A|odd- The little Weyl group permutes the 
Ei with even and odd indices separately and therefore equals the semisymmetric group. 
Finally, g = (^^^^, ^^-^, • • •)• Thus we can choose g = '^^{n — i)ei which shows r = 1. In 
particular, c^{x) is a multiple of R\{x] 1). 

Again, this can be made more explicit. Observe, that G is a subgroup of G := 
GL„_|_i(C) and V is the restriction of the natural G-action on A^C""*"^ to G. It is known 
(see e.g. [GW] Thm. 5.2.11) that as a G-module: 



(3.20) P = ^M^"+^ 



where r runs through all partitions with ri = r2, rs = r4, . . . and £(r) < n + 1. Now, we 
use again the GL^+i — GL^ branching rule. For A to be interlaced with r means Ai = ri, 
A3 = Ts, . . .. Thus, as a G-module, we obtain 



(3.21) P = ^M^ 



(n) 
A 
A 



where A runs through all partitions with i{X) < n. Here M^ is sitting in M^ ' where 
A* = (Ai, Ai, A3, A3, . . .). 

An element of V is represented by a skewsymmetric matrix A = (aij) of size n + 1. 
For / C [n + 1] of even size 2m let 



(3.22) Pf/(A) := Pfaffian (aij) ,e. 
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Then, the CapeUi operators for G corresponding to simple weights are 
(3.23) C^:= Yl PU{A)Pfi{dA). 



IC[n + l] 
\I\=2m 



Each G-module M^a™) decomposes into (at most) two components, namely M^^Jm-i) and 

(n) 

M^J^y Therefore, also Cm decomposes as Cm = -D(i2rn-i-) + D(^i2rr,-^ where 



(3.24) 15(12^)= Y, Ph{A)Pfi{dA), %2 



Y PU{A)Pfj{dA). 



\I\=2m 



IC[« + 1] 
n + lel ,\I\=2m 



For example, for n = 3 we get 



(3.25) D 



(1) 



d d d 



da 



14 



da 



24 



d d 

(3.26) Di^ii) = ai2— hasiTT 1-^23 



da34 
d 



dav. 



da3i 



da 



23 



(3.27) ^(111) = (ai2a34 - ai3a24 + 023^14) f - 



d d 



d 



d d 

+ 



d 



dai2 das4: dais da24 da23 dai4 
To describe the zonal spherical functions we identify V* also with skewsymmetric 

matrices of size n + 1 and pairing V xV* —^ C : (A, A*) \-^ ^ tr (AA*). The function E\ is 

a G-invariant functions on y © V* which is a joint eigenvector of the differential operators 

15(11) defined in (3.24) acting on the first argument. 

To make the Chevalley isomorphism explicit we define skewsymmetric matrices T{z) 

and T*: 



(3.28) T{z) :-- 



-S* - 

n,n 



■^n,n 







rp^ 














where S and S* are defined in (3.15). For example, for n = 4 we get (again setting 

Ui := zi - Z2 + zs - + . . . ± z^): 





/O 


-Z2 





-U2\ 






/o 


- 


1 0\ 











-Z2+Z3 


-Z4 


—U4 












-1-1 






(3.29) T{z) = 


Z2 Z2-Z3 











7 


T* = 


1 














Z4 


















1 











\U2 U4 








/ 






Vo 


1 


0/ 






while for n = 5 one has 


















/O 





-Z2 





-U2\ 






/O 


-1 0\ 







- 


-Z2+Z3 


-Z4 


—U4 









0-10 


(3.30) T{z) = 




Z2 Z2-Z3 


- 

^2-^3 


-Z2+Z3 




-Z4+Z^ 




-U5 




7 


T* = 




1 


0-1 





Z4 


Z4—Z5 















1 







\U2 U4 


U5 










/ 






Vo 


1 


0/ 
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Let B C G he the Borel subgroup which is the stabihzer of the flag {vi), {vi, V2), ■ ■ ■ 
where Vi = '^±<:j<:n^± Gj. Then one can verify that z 1-^ T{z) is the embedding of a* 
into V which is induced from v* = T* and the Borel subgroup B. Thus, we get that the 
restricted zonal spherical function E\{T{z),T*) is proportional to Rx{z; 1). 

The other interpretations of spherical functions are most interesting when n is odd. 
Then H = H* = Spn-i{C) and A \-^ E\{A,T*) is an S'ptj,_i(C) -invariant functions on 
the space V of skewsymmetric matrices of size n + 1 which is a joint eigenfunction for the 
differential operators Dniy Any invariant function is uniquely determined by its value at 
T{z) and we have Ex{T{z), T*) = Rx{z] 1). 

The open G-orbit in V is isomorphic to G/H = GLn{C)/Spn-i{C). Thus, the 
puUback of an SprL-i{C)-mva,na,nt function on V leads to a S'pn-i(C)-biinvariant func- 
tion on GLniC). Clearly, not all of them are of this form. For this, we have to make 
the function A \-^ Pf (^) invertible since its zero-set is the complement of the open or- 
bit. Since £'(i«)(A, T*) = Pf(A), this corresponds on a* to make the function 7r{z) := 
i?(in)(z) = Yli odd -^^ invertible. We can extend the definition of R\{z) to every element 
A G A' := {A G Z'^ I Ai > . . . > An} by Rx := ii-'^Rx+m{i-) for m » (by Corollary 2.4, 
this is independent of the choice of m). Then Rx{z; 1), A G A' is the radial part of an 
5'pn-i(C)-biinvariant function on GLn{C) which is a joint eigenfunction for all GLji{C)- 
biinvariant differential operators. A similar result holds if GLn(C) is replaced by SLn{C). 
Then a* should be replaced by {z E a* \ d{z) = 1} and A should be an element of A'/Z(l"). 

The case of G = GLp{C) x GLi{C) acUng on (C^^C) © (C^)*. 

The action of (A, s) G G on a pair of vectors (w, v) is {sAu, {A^)~^v). Here Ky is generated 

by £1 + e', — £p, and e' with degrees 1,1, and 2, respectively. Thus, if we put 

(3.31) ei = -£p, 62 = £1 + £p + e', 63 = -£i. 

then the generating weights become ci, 61+62, and 61+62 + 63. In particular, the degree 
of 6i is 1, 0, 1, respectively. The little Weyl group is generated by the permutation which 
swaps £1 and £p, and therefore 61 and 63. Thus, the Capelli operators are described by 
semisymmetric polynomials in n = 3 variables. The vector q = (^^, . . . , ~^^; 0) equals, 
up to a VTv-fixed vector (£^, 0, . . . , 0, -^; ^) = P^{ei - Ep + e) = 2^(26i + 63). 
This shows r = ^^. 

The concrete decomposition of P as a G-module has been worked out in [VS], see 
also [VK] §11.1-11.2. Here, we give only the fundamental Capelli operators. Denote the 
coordinates of F by wi, . . . , Up] vi, . . . ^Vp. Then 

21 



d d 

(3.32) 15(1) = J2v, — , 15(11) = $^«^^, ^(111) 

1=1 * i=l * 



(Vwit'*)('^^ 



a d 



^-^ dui dvj 



The zonal spherical functions have been investigated by Vilenkin-Sapiro [VS] (see also 
[VK] 11.3.2). They are eigenf unctions for the three differential operators -D(i), -D(ii), and 
-D(iii) above. For the Chevalley isomorphism we define 

u{z) = (z2,0, ...,0, Z2 - zs); 

V{z) = {Zi - 22 + ^3, 0, . . . , 0, -Zi + Z2); 

3.33 

< = (1,0,...,0); 

^o* = (l,0,...,0). 

Then z 1-^ {u{z),v{z)) is the embedding a* ^^ V which corresponds to (uq^Vq) G V* and 
the Borel subgroup which stabilizes the fiag (ei + 6^), (ei +6^, 62), ... , (ei + 6^, 62, ... , e^). 

To be eigenfunction for D^i) and -D(ii) simply means to be bihomogeneous in the u- 
and iJ-coordinates. Since H* = GL^_i(C) we get the following interpretation of zonal 
spherical functions: they are bihomogeneous GL^_i(C)-invariant functions on C"^ © (C"^)* 
which are eigenfunctions for the "Laplace operator" D^m). 

This interpretation has also a real form: The complexification of U{n — 1) is H* = 
GL„_i(C) while V is the complexification of C"^, considered as an R-vector space. The 
coordinate function Vi is then simply the complex conjugate Ui of Ui. Thus, E\{u^ u, Uq,Vq) 
is an t/(n — l)-invariant function on C^ which is bihomogeneous in the holomorphic and the 
antiholomorphic variables and which is an eigenfunction of the (now genuine) Laplace oper- 
ator -D(iii) = Xli dudu • ^^ ^^^^ form, the Ex have been studied by Vilenkin-Sapiro [VS]. 



4. Difference operators 

For A G /c" let T\ be the shift operator T\f{z 



% 



f{z — X). Let Ei be the z-th canonical 

For reasons of clarity we adopt the following notation: 

and T„ , := T2 



basis vector of k"^ and Ti :- 

Xi := Z2i-i, Vi := Z2i, T^^^i := 72^-1, and T^ j := T2i. Then we define the following block 

matrices whose entries are difference operators (where t is an indeterminate): 



(4.1) X(t) 



(4.2) 2)(t) := 



"^+t){x^+r 


f 


-J _ xf+^-'T 


x,i 


i=l...n 






i 


.. . \ 




J = 1 . . . fi 


^" j = l...ri 1 


\y^+r)^+l-J _ yf+^- 




i=l...n 




{y,+r)^-^ 


i = l...ra / 




-■ j = l...n 


-■ j = l...ri / 


(x,+r)--J' 


i=l...n 




(a;,+r)^-J' 


i -'-Xjt 


i = l...fi 1 


j = l...fT 




j = l._..n 1 




i 


i = l...n 
j = l...rl 


(2/^4 


-t) 


>,+r)^- 


-J 


n+l-j 

-VT 


Ti 


l,i 


i = l.^.n j 
j = l...n / 
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The semisymnietric Vandermonde determinant is: 

(4.3) ^{z) := Yl i^^-^j)= n (^*~^i) n (y^-yj)- 

l<i<j<n l<^<j<^ l^^<i^II 

j — i even 

Now we define the operators 

(4.4) X{t) :=ip{z)-^detX{t) and y(t) := ip{z)-^ det^{t). 

First observe, that the entries of X(t) and 2)(t) commute if they are in different rows. 
Thus, the determinants are weU defined. 

4.1. Lemma. Both X{t) and Y{t) act on V^ . 

Proof: Let / G V^ . Then both X{t)f and 2)(t)/ are polynomials which are skewsymmetric 
with respect to both factors Sfi, Sn of W. Therefore, they are divisible by ip{z) and the 
quotient is VF-symmetric. D 

4.2. Lemma. For f e V^ holds degX{t)f < degf and degY{t)f < deg/. 

Proof: We use the following elementary fact: let A = (aij) a matrix with entries in a filtered 
ring such that deg a^j < (i^ + d'j for integers d[ and d'j . Then degdet A < J^ii^-i + '^i')- 

We apply this to X(t). Using that the operator 1 — Tj has degree —1 the entries of 
X{t) have degree d[ + d'j with e := n — n and 

{d[,...,d'J ={ ,...,0,0, -£ , -£,...,-£) 
«,...,O=(fi-l,...,l,0,n-l,n-2,..., e) 

Thus degdet3e(t) < E^(^^ + O = deg(/?(2). 
For 2)(t) one argues in the same way with 

(rf'i, ...,<)=( ,. ..,0,0,1-5, 1-e,. ..,!-£) 

«, ...,0 =(n-l,...,l,0,n-2,n-3, ...,£-1) □ 

Next we derive an explicit formula for X{t) and Y{t). For / C {1, . . . , n} put £/ := 
Xlie/^^ ^^^ '^ifi^) ■~ fi^ ~ ^i)- Thus Tj = Yliei^i- With this notation we define 

(4.7) Di:= Hz, Y[(z^-z^-r) H {z, - z,)-' Tj. 

n — i even j — i odd 3 —'i- even 

Let Podd be the set of subsets / C {l,...,?i} such that there is a. w E W with 
/ = ty{l, . . . ,m} (where m = \I\). Thus / G Podd if and only if the number of its odd 
members is equal or one more than the number of its even members. Let Peven ^ Podd 
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consist of those sets where these numbers are equal. This is equivalent to |/| being even. 
Finally, we set 



(4.8) |/|o:= |{ze/|zodd}| 



l^/lodd- 



4.3. Proposition. We have 

(4.9) x{t)= j2 (-1)'"° n(^+^^)^^ 



iePo6 



igl 
i odd 



(4.10) Y{t)= Yl (-1)'"° n (t + ^^)Di 



lePo- 



i even 



Proof: Clearly both X{t) and Y{t) have expansions of the form "^jCiTi where / runs 
through all subsets of {1, . . . , n}. First we show that in Y{t) only the / G Peven contribute. 
For this, we subtract in (4.2) the columns 1, . . . , n from the columns n + 1, . . . ,n + n — n, 
respectively and obtain 



(4.11) det2)(t)=det 



{x,+r)^-^ 


i = l...n 




3 = 1. ..n 






i = l...n 


\y 




-1 


j = l...n 





-<-'- 



i^l . . .n 



\ 



{y^+t)iy^+r)^-' 



i^l . . .ri I 
j = l...n / 

Next, we use the well known fact that the determinant of a block matrix is 
(4.12) det(^ ^j=det{A-BD-^C)detD 

if D is invertible. In our case, the entries of A — BD~^C are linear combinations (over the 
field of rational functions in z) of 1 and the T^^iTyj which proves the claim. 

Since the given form of the operator Y{t) is VF-symmetric it suffices to check the 
coefficient of Tj = T^ i . . . T^; ;Ty i . . . Ty^i where / = {1, . . . , 2/}, / = 0, . . . , n. Every entry 
of the matrix (4.11) is of the form a + bT^^i or a + bTy^i. Thus the required coefficient is 
the determinant of the matrix obtained by replacing that entry by 6 if i < / or a if i > /. 
The ensuing matrix has the following form with dimensions as indicated: 



(4.13) 



/ O^xn *lxn \ 

*n — l X n ^n — l X n 

*l X fi U; X n 

\0 



-I X 



*n — l X 



We interchange the first with the third block of rows. Then the determinant gets multiplied 
by (—1)^ = (—1)'^'° and the matrix acquires block diagonal form. The blocks are, up to a 
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common factor in each row, in Vandermonde form. Thus the formula given in the Theorem 
is easily established. 

The case of X{t) is similar but a bit more complicated. Here we can subtract in (4.1) 
column number n + 1 through 2n — 1 from columns 2 through n, respectively. Then we 
obtain 



f[Ui. 



(4.14) detX(t) = det 



i=l...n 



{x^+t){x^+r] 



n-j 



3=2. ..n 



X 



n-j 



T 



V 



[Vt\i=l. 



yi -'-1 



y,i- 



j = 2...fi 



{yi+r)-~ 



j = l...n 



j = l...n 



with Ui 



{xi+t){xi+r 



,n-l 



xfTrc^i and Vi := (yi+r)— — yf-Ty^i. Arguing as above, one 



notes that all entries of A — BD~^C are linear combinations of 1 and T^^iTyj except those 
in the first column where also T^^i appears. Thus, if c/ 7^ then the number of odd 
elements is equal or one more than the number of even elements, i.e., / G Podd- 

To determine the correct coefficient we proceed as above. The case / = {1, . . . , 21} is 
the same. In the case / = {1,...,2/ + 1} one has to move in (4.14) the first column to the 
n + 1-st place (i.e., between the two other blocks). D 

The main feature of the difference operators is the following cut-off property: 

4.4. Lemma. Expand X{t) or Y{t) as '^jCj{z)Tj. Assume r 7^ 0. Then for any /U G A 
holds: if n — Ej ^ A then ci{g + n) = 0. 

Proof: Since r 7^ (and, as always, g dominant), the denominator of cj does not vanish 
aX g + n- If fx — Ej ^ A then either //^ = and n G / or there is an i < n with fXi = Hi+i 
and i G /, i + 1 ^ /. Now we use the precise form of cj{z) established in Proposition 4.3. 
From the definition of Dj, (4.7), we obtain that c/ is a multiple of Zn{zi — Zi^i — r), hence 

c7(^ + /u) = o. n 

Combining all results, we obtain the main result of this paper. 

4.5. Theorem. Every R\, X E A, is an eigenvector of both X{t) andY{t). More precisely: 

(4.15) X{t)Rx = n (^ + ft + A^) ■ Rx 

i odd 

(4.16) Y{t)Rx= n {t + g^ + X^)■Rx 

i even 

Proof: We may assume r 7^ 0. The case r = then follows by continuity. Let R := X{t)R\. 
Lemma 4.2 implies degi? < degRx = |A|odd- Let /U G A with |/u|odd < |A|odd and /U 7^ A. 
If X{t) = Y^j ci{z)Ti then R{g + ^) =Y.i ci{q + Ij)R\{q + ifJ' - £/))• If /U - £/ G A then 
Rx{g+ (n — Ei)) = by definition of Rx. Otherwise, cj{g + n) = by Lemma 4.4. Hence, 
R{g + n) = which shows that R is a, multiple of Rx- The coefficient of T^ in X{t) is 
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c{z) = nr=i(^ + ^2i-i)- Thus, evaluation in z = g + X gives R = c{g + X)R\. The same 
argument works for Y(t). D 

4.6. Corollary. Let 

(4.17) 

Y{t)=t^ + Yit^-^ + ... + Y^. 

Then Xi, . . . , X^, Yi, . . . , 1^ are pairwise commuting difference operators. 

Example: We compute Xi and Yi. Any contribution to the coefficient of t'^~^ in (4.9) 
comes from I = $, I = {i} with i odd, and / = {^, j} with i odd, j even. Since D0 = 1 we 
get 

(4.18) x,= j2z,-j2 ^{0 - E ^i^^}- 

i odd i odd * odd 

J even 

Similarly, in (4.10), the contribution for t-~^ comes from / = and / = {i,j} with i odd, 
j even. Thus, 



(4.19) y-i = E ^^ - E ^{^.. 



i,j}- 



i even * odd 

j even 



The operators Xi, . . . , Xfi, Yi, . . . ,Yn defined in (4.17) generate a polynomial algebra 
TZ C EndP^. We show that it is canonically isomorphic to V^ . More precisely: 

4.7. Proposition. 

a) Every element D ElZ has an expansion 

(4.20) 15= 5] cJ'(^)T^, 

where \l/o is the smallest W -stable suhmonoid of Z" containing A and where the 
coefficients c^ are rational functions in zi,...,Zn with poles along the hyperplanes 
Zi — Zj = a where i — j j^ is even and a G Z. 

b) The coefficient Cq{z) is in V^ and the map IZ -^ V^ : D 1-^ Cq [z] is an algebra 
isomorphism. 

c) For every D e TZ and A G A holds D{Rx) = Cq{q + X)R\. 

d) If r ^ Q the coefficients c^{z) have the cut-off property: Let A G A with X — n ^ A. 
Thencj^{g + X) = 0. 
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Proof: a) Since \l/o contains all VF-translates of elements of A it contains all £/ with 
/ G Podd- Hence, by Proposition 4.3, the generators Xi, . . . , X^, Yi, . . . , 1^ of 7^ have an 
expansion as claimed. This implies the result easily for all D eTZ. 

b) Since \l/o is contained in N^ it is a pointed cone, i.e.. A, /U G \I/o with A + /U = 
implies A = // = 0. Looking at how two operators with an expansion as in (4.20) multiply 
this implies that D ^-^ Cq is an algebra homomorphism. It is an isomorphism, since the 
generators Xi and Yi of IZ are mapped to free generators of V^ . 

c) The assertion needs to be checked just for the generators of TZ and there it is the 
content of Theorem 4.5. 

d) The algebra 7^ acts on the dual space (P^)* on the right by {5D){f) := 5{D{f)). 
Let V = (vi) E k"^ such that Vi — Vj ^ Z whenever z — j 7^ is even (e.g. v E g + A, 
since r ^ Q) and let Sy : f ^-^ f{v) be the corresponding evaluation function. Then 
SyD = '^ij_cj^{v)dy-^. Thus the cut-off property is equivalent to the statement that 
Q)\eAkSg-^\ C (jy^y is 7^-stable. It suffices to check this for generators of TZ which is the 
content of Lemma 4.4. D 

Next we study the monoid \&o more closely. 

4.8. Lemma. The monoid \l/o has also the following descriptions: 

a) It is generated by {si \ i odd} U {si + Ej \ i odd, j even}. 

b) It consists of all A G N'^ with [A]i > 0. 

Proof: a) Since A is generated by all £{i,...,m}7 m = 1, . . . , n, the monoid \l/o is generated 
by all £/, / G Podd- But those can be obtained from the given subset. 

b) Let \l/o be the set of all A G N"" with [A]i > 0, i.e., |Aodd| > |Aeven|- We have to show 
\l/o = ^I/q- The inclusion \l/o C \1/q follows, e.g., from a). For the converse, let A G \I/o. We 
show A G \E'o by induction on |A|odd- If |A|odd = then also |Aeven| = 0. Thus A = G \I/o. 
For |A|odd > there are two cases. If |Aodd| > |Aeven| then choose any odd i such that 
Xi > 0. Then A' := A — Ei is in \l/o hence, by induction, in \l/o. Thus also A = A' +£i G \I/o. 
If |Aodd| = |Aeven| > then there is i odd and j even such that Ai > and Aj > 0. Then 
X' := X — Ei — Ej is in \1/q, hence in \l/o by induction. We conclude A G \I/o, as well. D 

In the theory of symmetric polynomials, the containment relation A C ^u for A, /U G A 
is defined as /U — A G N"^. The semisymmetric analogue is // — A G \E'o or, equivalently, 

(4.21) A C ^ M^ A C ^ and [A]i < [ij]i. 

Example: We have (1, 0, 0) C (1, 1, 0) but (1, 0, 0) g (1, 1, 0). Moreover, we have 

(4.22) (1,0,0),(1,1,0)C(2,1,0),(1,1,1). 
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This implies in particular that (1,0,0) and (1,1,0) have no supremum. Therefore, as 
opposed to the classical containment relation, its semisymmetric analogue does not form 
a lattice. 

Now we prove that the polynomial Rx vanishes at many more points than it is sup- 
posed to by definition (Extra Vanishing Theorem). 

4.9. Theorem. For A, /U G A holds Rx{g + n) = unless A ^ ^. 

Proof: We may assume r ^ Q since the general case follows by continuity. For fixed A let // 
be a counterexample (i.e., R\{g + n) ^ and A ^ /u) which is minimal with respect to "^". 
Since p + /U is not in the VF-orbit of ^ + A there is D G 7^ such that Cq{q + A) j^ Cq {g + n) . 
From D{R\) = Cq'(^ + X)Rx we obtain, after substituting z = g + fi: 

(4.23) {c^{g + X)-c^{g + i^))Rx{g + //) = J2 c^{g + l^)Rx{g + l^ - v) 

If /U — ?7 G A then Rx{g + A* ^ ^) = by minimality of fx. Otherwise c^{g + /u) = by 
Proposition 4.7 d). Contradiction. D 

As an application we derive an explicit formula for Rx when A is a particular kind of 
"hook". 

4.10. Corollary. Let a, m > 1 be integers with m odd. Then 

(4.24) i?(„i™-i) = (i?(i) - l)(i?(i) - 2) . . . (i?(i) - a + l)i?(i^). 

Proof: Denote the right-hand side by /. Let A = (a l"^~i) and /U G A with |/u|odd < |A|odd = 
a+^^2cii and f{g+ij) 7^ 0. Then R{^i-m.-){g-\- (i) 7^ and i?(i) (^-|-/u) 7^ 1, 2, ... , a— 1. The Extra 
Vanishing Theorem 4.9 implies (1"^) C ^, hence Urn > 1 and [/u]i > [(l"^)]i = 1. From 
-R(i) (^ + A*) = [a*] 1 (Corollary 2.8) we obtain [n] 1 > a. Thus ni = [/u] 1 + {^2 — A^s) + • • • ^ '^■ 
We know already 1U3, /U5, . . . , /U^ > 1. Since |At|odd < a + ^^^^, equality holds throughout. 
This implies easily fi = X. Therefore, / must be a multiple of Rx- Equality follows from 
the fact that the coefficient of z^'^'^ = zfzsz^ ... ^m is 1 in both cases. D 



Remarks: 1. For even m, the polynomials R(^ai"^-'^) will be calculated in Corollary 9.5. 
2. For m = 1 one obtains in particular i?(„) = i?(i)(i?(i) — 1) . . . (-R(i) —a + 1) which clearly 

of the complete symmetric functions. 



do not generate V^ . Therefore, the polynomials R(a) are not a semisymmetric analogue 
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5. The top homogeneous components 

The highest degree components Rx{z; r) of Rx{z; r) are also of high representation theoretic 
interest (see Theorem 3.4). We show that they are eigenfunctions of differential equations. 
More precisely, put 



(5.1) X(t) := 



/ 



V 



x'l ^ {xidxi+{n-j)r+t) 



,11-3 



{yidy, + {n+l-j)r) 



i = l. 


.n 


J-1- 


.n 


J-1. 


.n 

.7% 



n-j 



— X^ 
n—j 






i=^l . . . n 
3 = 1... n 



(5.2) 2)(t) 



/ 




'xr 


i = 
3 = 


l...n 
l...ri 


v 


- 


r' 


-j 


i = l...n 

j = l.^.n 



yf~^ {yidy, + {n-j)r+t) 



2—1. . .TL 

j — l...n 



i^l . . . n 
j = l...n 



and X{t) := ip{z) ^ detX(t), Y{t) := ip{z) ^ det2)(t) where d^^ = d/dxi and ^y^ 
These are linear differential operators with rational coefficients. 



d/dyi. 



5.1. Theorem. Every R\, X E A, is an eigenvector of both X{t) andY{t). More precisely: 
(5.3) X{t)Rx= W {t + g, + X,)Rx 



i odd 



(5.4) Y{t)Rx = n (^ + ^^ + -^*) ^^ 



Proof: Let f E V is homogeneous of degree d. For each entry aij of X(t) or 2)(t) we 
know that aij{f) is a polynomial of degree < d + d[ + d" (with d[, d" as in (4.5) or (4.6)). 
Using the fact that 

(1 — Tj:^i)f{z) =dxif + lower order terms 



(5.5) 



(1 — Ty^i)f{z) =dy-f + lower order terms 



one easily calculates that the d + d[ + d'j-degree component of aij{f) is dij{f) where dij 
is the ij-entry of X(t) or 2)(t), respectively. Since J^ii^i + d") = degLp{z), we get that 
X{t)f or Y{t)f is the (i-degree homogeneous component of X{t)f or Y{t)f , respectively. 
Now the assertion follows from Theorem 4.5. D 

Remark: The operators X{t) and Y{t) are the semisymmetric analogues of the Sekiguchi- 
Debiard operators, [Se], [De], which characterize Jack polynomials. 



If we expand X{t) and Y{t) as a polynomial in t. 



(5.6) 



X{t) 



t^ + Xit^-^ + . 



+ x, 



Y{t) =t^ + Yit^-' + ... + Y 
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we obtain as in Corollary 4.6 pairwise commuting differential operators Xi, . . . , X^, Yi, . . ., 
Yn with Rx as common eigenvectors. In general, these operators seem to be more difficult 
to compute explicitly than their difference counterparts. We give a formula for the most 
important ones, namely those of order one. For odd i we define the following rational 
function: 

n (z.-zj) 

/r ^\ J <^^'^" u f Zi for n odd 

(5.7) Ui-.^Vi — :^ — where -^i := <^ / 

II [Zi — Zj) 11 for n even 

jy^i odd 



5.2. Theorem. The following equations hold: 

_d_ 

dz. 

(5.9) ?7':=Xi-Fi-nr= ^ w,^ 

I odd 



— d 

(5.8) T] := Xx — nnr = y ^Zj— — (Euler vector field) 



Moreover, for all X E A holds rj{Rx) = |A|odd-RA; v'i^x) = [-^]i-Ra- 

Proof: Let E be the Euler vector field. By (5.3), we have Xi{Rx) = |^ + A|odd-RA- From 
|A|odd = degi?A and |^|odd = ^IW it follows that t] — E kills every Rx and therefore 
every VF-invariant. The (non-symmetric) polynomials are all algebraic functions of the 
semisymmetric ones. Since r] — E is a derivation, it kills all polynomials, i.e., i] — E = 0. 
By (4.18) and (4.19), we have 

(5.10) E' := Xx-Yx = [z]i-Y, ^{^} = Ni " E <'^^ 

i odd i odd 

where 

n (Zi - Zj - r) 

/ j even 



(5.11) u[:=v. 



n (zt - Zj] 

j=/^i odd 



From (4.15) and (4.16) we obtain E'{1) = [q]i = nr. Thus, [z]i — J2i odd'^'i ~ ^'^ ^^^ ^^ 
get 

(5.12) Xx-Yx-nr=J2 <i^ - ^i)- 

i odd 

This implies (5.9) since (1 — Ti){f) = ^ + lower order terms. D 

The derivations ry, rj' induce a bigrading on V^ . More precisely, for integers a, b let 

(5.13) V^, := {/ G P^ I vif) = af. v'if) = bf}. 
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Then V"^ = ©a>&>o^^- To describe P^ explicitly, we have to find bihomogeneous 
generators of V^ . For this, we introduce the semi-symmetric analogue of the elemen- 
tary symmetric polynomials, namely e^(z) := R(^im^(z;r). More explicitly we have by 
Corollary 2.9: 

(5-14) ( ^ _ ( ^ _T 

Now, we consider the basis of V^ which consists of all monomials in the e^,- More 
precisely, we define* for any A G A 

(5.15) e^ := e^-^^e^^"^^ . . . e^ri^-^"e^. 



This parametrization is chosen such that the leading term of e\ is z^'^^K Then V"^ 
spanned by all e\ with |A|odd = « and [A]i = h. 



b IS 



e^ 



5.3. Corollary. For A G A consider the expansion R\ = Xlu'^A^e^. Then only those e^ 
occur for which |/u|odd = |A|odd and [/u]i = [A]i. 

Remark: This result will be generalized in Theorem 6.6. 

We use Corollary 5.3 to compute R\ for all two-row diagrams. We use the multinomial 
coefficient {^^^"^^^kj ■= ki\"',[kj. where a = h + . . . + kn- 

5.4. Theorem. For integers a > b > let Cab = (5) (~aO ■ ^^^'^ 

(5.16) B,„„ = i5:(-2'-)( "> ) 

where the sum runs through all n E A with |/Uodd| = ^ o^nd \neven\ = b. 

Proof: We use a result for the usual Jack polynomials Px{z;r). Stanley ([St], see also 
[KSl] Prop. 3.4 for a proof in the spirit of this paper), has shown that there is a generating 

series 

00 

(5.17) 5^z;.P(a)(^;r) = n(l + ^0-^ 

a=0 i 

where the Va = (~^) 7^ 0. Then, by the Comparison Theorem 2.7 (with /i = (a, 0, . . .)) 
there are constants Wa,b 7^ such that 



(5.18) Yl Wa,bRiab)iz) = 11(1 + 



z.)-'"-. 



i>b>0 i odd 



* In this notation, one has to be careful to distinguish between e^ (the index is a number) 
and e(a) (the index is a partition). The latter equals e". 



31 



Now, we expand the right-hand side in bihomogeneous components. For this observe 

(5.19) l[il + z,) = l + J2 e^(^odd) = 1 + 5^ e,(; 



i odd i>l i>l 



Thus, we get 



J2 ^a,bRiab)iz)=J2{ /){J2''^ 
a>b>0 d ^ ^ ^ i>l 



•^1 ^2 • • • ^n 



e/x 



Now, we compare the bihomogeneous components of bidegree (a, a— 6) of both sides and get 
formula (5.16) up to the scalar Cab- But that scalar is easily obtained by the requirement 
that the coefficient of e^^^) should be 1. D 

Examples: 1. The case n = 3. The summation in (5.16) runs through all /U G A3 with 
A*i+A*3 = '^ cii^d /U2 = h. If we put /U3 = /c, we get n = {a — k, 6, k) with < /c < min(a — 6, h). 
Thus, 

/p-rjl^ D Sr^ \a-b-k,b-k,k)\a-k) sr^, ^\k\ k I \k) ^ 

{b.Zi) Ka,b,0 - 2_^ .a\ (-2r\ ^a-k,b,k - 2_^[- i-) ^a+2r-l\ ^a-k,b,k 

k [b)[ a ) k [ k ) 

Now, the recursion formula (2.10) implies Ra+i^b+i,c+i = ^^Ra,b,c- Thus, we obtain a 
formula for R^ for arbitrary |U G A3: 



(5.22) i?^^,^2,^3 = 2_^(-l) /^^_^3+2r-n ^1' 



/'A'i-A'3+2) — 1\ 



-IJ-2-k U2-IJ-3-k U3+k 



'2 "=3 

A; 



with 

(5.23) 61=^1-^2+^3, 62=^2, 63 = ^12:3. 

This formula can be rewritten in two ways. First, as a hypergeometric function: 

(5.24) i?^,,„^3-ei 62 ^3 ■2Fi(^^^_^^_2r+l ^ 

Secondly, we can express the sum (5.22) as a Jacobi polynomial which is defined as 

(5.25) P-/^(.):=f" + "V2i^if""'"^"^^^' ''"^ 

\ n J V a + 1 
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For this, we invert the order of the summands. Since k runs from to the smaUer of fii — fi2 
and H2 — fJ^s we have two cases. Set fx = (/ci + /c2 + ^3, ^2 + ^3, ^3)- Then the first case is 
ki < k2- The substitution k = ki — I gives 

where {a)i = a{a + 1) . . . (a + / — 1) is the Pochhammer symbol. Thus: 



(5-27J ii; e2 63 -2^11 , , . ^ 
\fci/ \ ki J \ k2 - ki + 1 


eie2\ 
63 / 


This, and a similar computation for ki > /c2 gives 


\ fci / 2 3 fci \ _ 


e2s f 
— ) fo 



for ki < /c2 
(5.28) RM = \ /' 

(-fci-2r) ^e^-'=^e^^+'=^■P£-^^'2'^-^(l-2^i^) forA;i>A;2 

V ^3 

These formulas are essentially due to Vilenkin-Sapiro [VS], see also [VK] 11.3.2. 

2. The case n = A. In this case, we put ^2 — fJ'S = k and fX4 = I. Then ^ = [a — k — l,h 
l,k + Ij I) and we get 

- _ 1 'sp /^ a-k-l \f-2r\ 

Ra,b,o,o - —X.ya-h-k,h-k-2l,kj) \a-k-ir^ 
(5.29) 



=5: 



{-a+h)k {-h)k+2l ^a-b-k^b-l-2l^k^l 



where 

ei = ^1 — ^2 + ^3 — ^4? 62=^2 + ^4? G3 = ZiZs — ^2^4? G4 = ^2^4- 

This can be expressed in terms of one of Horn's hypergeometric functions (see e.g. [Ba] 
§5.7.1): 

(5.30) Ra,b,o,o = er'el-Hs{-b,-a+b,-a-2r+l;^, ""' ^ 



62 6162' 
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6. Triangularity 

In this section, we investigate vanishing properties of the coefficients of R\{z; r). For this, 
we consider the inhomogeneous dominance order: for (U, A G N" define 

(6.1) fJ' < X -^^ Hi + . . . + Hm < Ai + . . . + Am, for all ?n = 1, . . . , n. 

The homogeneous dominance order, commonly considered in the theory of symmetric 
functions, is 

(6.2) A* = -^ ^^=^ A* ^ -^ ^i^d \n\ = |A|. 

Recall, that we defined the leading term of Rx as z'-^^ where [A] is defined in (2.3). The 
next theorem justifies this terminology. 

6.1. Theorem. For every A G A there are expansions 



(6.3) Rxiz)= Yl ^W"* «^^ Rx{z)= Yl 



a\ij,z^- 



Proof: For 1 < m < n and f E V denote the total degree of / in ^i, . . . , ^m, by deg^ /. 
Let m := [m/2j and fn := m — m = \m/2\. We show first deg^ X{t)f < deg^ / and 
deg^ Yit)f < deg^ / for all / G P^. 

Lemma 4.2 is nothing else than the case m = n. The general case is the same except 
that the entries in the rows involving Xfn+i, . . . ,Xn and ym+i, • • • , y n have degree 0. Thus 
the degree of X(t) can be computed by taking in (4.5) or (4.6) the m largest entries of the 
d'^ and the entries of d[ which correspond to a; i, ... , x^, yi, . . . , y^n. Thus 



(6.4) deg^ det X{t) = y^{n — i) + /^("^ ~ + EkilL — n) = /^(?^ — i) + / X 

m rn 

(6.5) deg^ det 2)(t)= y^{n — i) + /^{^ — z — 1) + m{n — n + 1 

j=l i=l 

fn rn 

= ^(n-z)+ J^(m-z). 



i=l 1=1 



On the other hand deg^ Lp{z) = X]ili(^ ^ + Yl'iLiilR ~ which proves the claim. 

For A G A let Vx and V^ be the intersection of V^ with the span of all z^ with 
fx < [A] and ^ < [A] , respectively. Then, by what we have proved above, both Vx and V^ 
are stable under X{t) and Y{t). The monomial symmetric polynomial m[x]{z) is in Vx 
but not in V^. Thus, V^ is of codimension one in Vx- Because the action of X{t), Y{t) is 
diagonalizable there is exactly one z/a G A such that Ri^^ is in Vx but not in V^. 
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It remains to show ux = X for all A. If there exists a counterexample then choose 
one which is minimal with respect to the order relation [u] < [A]. Since Ri^^ contains z^"^' 
(Lemma 2.3) we have [ux] < [A]. Thus, by minimality, R,^^ G Vi,^ C Vx in contradiction 
to the definition of u\. D 

Examples: 1. If A is of the form (a, a, 6, 6, c, c, . . .) then we know from Theorem 2.6 
that Rx is a polynomial in the even variables Z2, z^, . . . only. This can also be seen from 
triangularity: since [A]i = we have /Ui = for every z^ which occurs in Rx- Hence zi 
does not occur. By symmetry, no odd variable occurs. 

2. If A is of the form (a, 6, 6, c,c,.. .) then [A] = (a, 0, 6, 0, . . .). Hence triangularity 
prohibits, e.g., the occurrence of monomials Zi^Z2^ . . . with /Ui + /U2 > a. 

This form of triangularity seems to be optimal when the expansion of Rx in monomials 
is considered but, since monomials are not bihomogeneous, it does not cover the bigrading 
result of Corollary 5.3. Therefore, we expand Rx in elementary semisymmetric symmetric 
functions e^ defined in (5.15). Then, an equivalent form of Theorem 6.1 is that for every 
A G A there are expansions 

(6.6) Rx{z) = ^ ax^e^ and Rx{z) = ^ aA^e^. 

The point is now to define an order relation on A which is stronger than [fj] < [A]. 

For this, let $+ C Z"^ be the submonoid generated by all simple roots Si — £^+2, 
1 < i < n — 2. Recall from Proposition 4.7, that \l/o C Z"^ was defined to be the smallest W- 
stable monoid containing A. We define the semisymmetric analogue of the inhomogeneous 
dominance order on A as 

(6.7) ^^X 44 A-;UG*i :=^o + $+ 

6.2. Lemma. The monoid \l/i has also the following descriptions: 

a) It is generated by {si — £^+2 \l<i<n — 2}U {Sn-i + £n, £2n-i}- 

b) It consists of all A G Z"" with < Aodd; < Aeven; Cind |Aeven| < |Aodd|- 

Proof: a) By Lemma 4.8, the monoid \l/o is generated by all elements of the form Ei + Sj, 
{i odd, j even) and e^, (z odd). Using the generators of $+, one can obtain all these 
generators from either £n-i + £n or £2n-i alone. This shows the claim. 

b) First observe that the set of generators E in a) forms in fact a linear basis of Z". 
Now consider the set E' consisting of the linear forms Ai, A2, A1 + A3, A2 + A4, A1 + A3 + A5, . . . 
and Ai — A2 + A3 — \- . . .. Then the conditions in b) can be rephrased as £{X) > for all 
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£ G E'. Observe that E' contains |Aodd| which is a sum of two other elements, thus 
redundant. When we remove it from E' we obtain a set E* which turns out to be the dual 
basis of E. Thus ^i equals the set A G Z'^ with £(A) > for alH G E*. D 

Since [A]i = |Aodd| — |Aeven|, we have in particular 

(6.8) // ^ A ■^=^ //odd < Aodd, /"even < Aeven, and [//] i < [A] i . 

Next, we compare /U ^ A with [fj] < [A]. 

6.3. Lemma. The monoid \E'i := {A G Z" | < [A]} is generated by 

(6.9) {£, - ei+2 I 1 < i < n - 2} U {e^-i + £n, -£2}- 

Proof: One easily checks that the proposed set of generators is the dual basis to 

(6.10) {[A]i + ... + [A]^ I l<m<n}. □ 

The new order relation is indeed stronger than the one considered before: 

6.4. Corollary. /U ^ A implies [//] < [A]. 

Proof: We have -£2n = -£2 + (£2 - £4) + • • • + (£2n-2 - £2n)- Since 2n (resp. 2n - 1) is 
the largest even (resp. odd) integer in 1, . . . , n we have £2n-i = — £2n + (£n-i + £n)- This 
implies \I/i C vEf^^ which is equivalent to the assertion. D 

The homogeneous version of "^" is defined as 

(6.11) // ^ A <^ H ^ X and |/u|odd = |A|odd 

Since |A|odd = for all A G $"•" and |A|odd > for all A G \I/o \ {0} the definition of /U ^ A 
simplifies to A — /U G $"'". Thus, we get 

(6.12) /U ^ A ^=^ //odd ^ Aodd and //even ^ Aeven- 

Now we are looking at expansions of elements in V^ in the form ^ ci\^e^. For 



^w ^ , , 

technical reasons we need a version which works for all elements in V. 



6.5. Lemma. For f G V^ and A G A the following statements are equivalent: 
a) In the expansion 

(6.13) f{ui+U2,U2,U3+U4,U4,...)= ^tt^W^ 
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(where Wn+i := if n is odd) only monomials u^ with n < [A] and //even < ^even 
occur, 
b) There is an expansion 

(6.14) f{z)= J2 ^A^^A*- 

Proof: "6J^a/': Let deg^ / be the total degree of / in wi, . . . , Um which is the same as 
the degree in zi, . . . ,Zm- Then one calculates 



m 



(6.15) deg^e;. = ^[A] 



^m/2 



Zlili ^2i-i if m is even, 



' ] Y-("^-l)/2 



1=1 



E-=r '' A2. + [A]i if mis odd. 



We conclude that /U ^ A implies deg^ e^ < deg^ e^. Thus, if u^ occurs in / then 
(6.16) ^1 + ... + ^m = deg^u^ < deg^f < deg^eA = [A]i + ... + [A]^, 

i.e., // < [A]. 

Now let deg^ be the total degree of / in W2, u^, . . . , W2m- Then, due to cancellations, 
one has deg^ e^ = X^^^i A2i. The same reasoning as above implies //even < Aeven whenever 
-u^ occurs in f{u). 

"aj^6/': Assume e^ occurs in the expansion of /. Then, by the calculations above, 
we have to show deg^ e^ < deg^ / and deg^ e^ < deg^ / for all m (actually is suffices to 
consider in the ffist case only m = 1 and all even m). 

We treat deg^ first. For this define a total order on the monomials w^: ffist we order 
them by deg^ and then by the lexicographic order on (Ai, A3, ... , A2, A4, . . .). One checks 
that Cp has the leading monomial uiUs . . .Up if p is odd and U2U4 ... Up if p is even. Thus, 
the leading monomial of e^ is u^^'. This shows in particular that if the leading monomials 
of e\ and e^ coincide then A = /t. Therefore, if there were an e^ occurring in / with 
deg^ e^ > deg^ / then take a maximal one. Its leading monomial w^ would not cancel 
out and would satisfy deg^ u'^ > deg^ / in contradiction to a). 

For degj^ we argue similarly. This time the total order on the monomials tt^ is by 
lexicographic order on (deg^ ti'^, A2, A4, . . . , Ai, A3, . . .). Then the leading term of e^ is 
U2U4 .. .Up if p is even and U2U4 . . .Up-iUp if p is odd. Hence the leading term of e^ is 
Ui^~ ^tt2^'^3^~ * • • -^n"- These terms are again distinct for different e^'s. The rest of the 
argument is as above. D 

Now we can state the better triangularity result announced earlier. 

6.6. Theorem. For every A G A there are expansions 
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a) R\{ui + U2, U2, Us + -U4, ^4, . . .) = ^ aX^U^ where ^ < [A], ^Ueven < Aeven/ 

Proof: By Lemma 6.5, it suffices to prove a). Ttie degree function deg^ (see last proof) is 
invariant under upper triangular linear coordinate transformations. Thus // < [A] follows 
from Theorem 6.1. 

To prove Heven < Aeven recall that the total degree of f{ui + U2,U2jUs + W4, W4, . . .) 
in the coordinates ^2, w.4, . . ■U2m is denoted by degj^ /. We show first that the operators 
X{t) and Y(t) preserve deg^. 

The substitution Z2i-i -^ U2i-i+U2i corresponds to Xi -^ Xi+yi and T^ j -^ Ty^iT~l in 
X(t) and 2)(t). The same reasoning as in the proof of Theorem 6.1 shows that deg^ det X(t) 
and degdet2)(t) are bounded by degi/? + m. Thus we have to find a way to decrease this 
estimate for the degree of detX(t) and det2)(t) by m. 

The idea is to add a multiple of the y^-row to the Xi-row. Since the entries are in a 
non-commutative ring some care is advised. For this, we develop detX(t) and det2)(t) as 

(6.17) Yl ± ^^^ ^nSt' • ■ ■ det A'P^'^jl'^ det ^-+i--->-+i+-v..,n 

where det Ay is the minor with row index in U and column index in V and where the sum 
runs through all partitions {1, . . . , n} = {zi, ji}U . . . U{irmJTn}^S. The degree of the last 
factor (involving 5") is zero. Now we show that the degree of each 2 x 2-minor is one less 
than expected which would prove the claim. 
For this we write 

(6.18) det A^^"^^' = det ( ^^ ^^ | = (xn + ax2i)x22 - a^2i(a:^i2 + "2:22) - [a, a^2i]a:22 

\ 3J21 3J22 / 

where 

{T^^i if A = X(t) and n is even 

yiT^^i if A = X{t) and n is odd 

y^^Tx^i if A = 2)(t) and n is even 

Tx^i if A = 2)(t) and n is odd 

As mentioned above this amounts to add a times row ^n + I to row ^/ of (4.11) or 

(4.14), respectively. Then it is easy to check that deg^(a:ij + ax2j) < deg^xij — 1 and 

deg^[a, X21] < deg^ xn — 1 which proves the claim. 

The rest of the proof is the same as for Theorem 6.1. For A G A let Vx be the space of 

all semisymmetric functions / in which only monomials u'^ with n < [A] and Heven ^ Aeven 

occur. Let V^ the same with additionally /U 7^ A. Then both spaces are stable under X{t) 

and Y{t). Moreover ex eVx\ V^. We conclude as in Theorem 6.1. D 
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Examples: The improvement of strong triangularity over the weak one is the more sig- 
nificant the smaller Aeven is- The most extreme case is A = (a) where Theorem 6.1 doesn't 
give any restriction. But Theorem 6.6 states R(^a) = XliLo ^i^ii) which is of course also 
a consequence of the direct calculation in Corollary 4.10. A more specific example is 
A = (5, 2, 0, . . .) for n > 10. In that case, Rx has, according to Theorem 6.1, 70 indepen- 
dent coefficients while Theorem 6.6 boils that down to 27. 

Remark: Part b) of the theorem is entirely analogous to a similar theorem for (shifted) 
Jack polynomials but part a) is a bit strange since the pretty asymmetric coordinates Ui 
appear. A conceptual explanation for their appearance would be very desirable. 

Now we can prove a triangularity property which is completely intrinsic for the polynomials 
Rx: 

6.7. Theorem. For every A, /U G A consider the expansion R\R^ = Y1it^\u.-^t- Then 
a\ = unless A, /U C r ^ A + /U. 

Proof: First we show r ^ A + // whenever a'^ ^ 0. We have 

(6.20) RxR^,= ^Y^hxr^h^r^Gr^Qr^ = ^Y^c^^v with C^= ^ bxrib^r2- 

Moreover, t\ < \ and T2 ^ H imply u = ti + T2 ^ X + H- Now, observe that the 
transformation matrix 6a^ is upper unitriangular. Thus, its inverse matrix has the same 
property, i.e., we have expansions e> = XIu^a ^v-^m- Hence 

(6.21) RxR^ = Y, ^XrRr = Yl «V^- 

UjT T 

with r ^ z/ ^ A + /U. 

Now we show A ^ r whenever a'^ ^ 0. The relation fj, ^ t follows then by symmetry. 
Let To be a C-minimal counterexample. Since A ^ tq, the Extra Vanishing Theorem 4.9 
implies -Ra(^ + to) = 0. Hence J2t'^XtRt-{q + tq) = 0. Again by the Extra Vanishing 
Theorem, only those r with t ^ tq contribute to this sum. For those we have A ^ r. The 
minimality of To implies t = tq unless a\ = 0. From this we derive the contradiction 
<^ro(^ + ro)=0. D 
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7. The binomial theorem 

In this section we derive a binomial type theorem for semisymmetric functions. The proof 
is similar to that for symmetric functions in [Ok]. 

So far, we considered values of Rx in the points z = g + X. Now we use that the 
difference operators also have a dual vanishing property. 

Recall that TZ is the algebra generated by the Xi and Yi where X{t) = "^^Xif 
and Y{t) = J^i'^i'^^- We introduce a degree function on TZ by letting deg^i = and 
degTA := |A|odd- Thus, degX^ = deg^^ = i. Observe that (P^)* is a right EndP^- 
module, hence a right 7^- module. For t E k^ let Sj- G (V^)* be the evaluation map 
/ '"^ /(''")• Then, as explained in its proof. Proposition 4.7 d) amounts to Q)\eAkdg+\ 
being an 7^-submodule of (V^)*. Now, for any a E k, let a := («,...,«) G /c" and 
Qa ■= Q + a = {{n — i)r + a)i. Then we have 

7.1. Proposition. Assume r 7^ 0. Consider the space M := (B\eAkS-g^-x. 

a) M is an IZ-suhmodule of iV^)* . 

b) Define a filtration on M by putting degd-g^-x := |A|odd- Assume a ^ —N — N ■ 2r. 
Then the map TZ —^ M : D \-^ d-g^D is an isomorphism of filtered k-vector spaces. 

Proof: Let A G A and D either Xi or Yi. Then, the assumption r 7^ makes sure that 
d-g^-xD can be computed in the obvious way since then the denominator of D does not 
vanish at — pa — A. 

a) It suffices to show that 6-g^-xDi G M for any /C{l,...,n}. This is no problem 
if A + £/ G A. Otherwise, there is an index j with j ^ /, z := j + 1 G /, and A^ — Xj. But 
then the factor Zi — Zj — r in Di vanishes at z = —Qa — X. 

b) The map clearly preserves filtrations. Since corresponding filtration spaces on both 
sides have the same dimension, it suffices to show surjectivity. We do that by induction 
on the degree. By the explicit formulas (4.9), (4.10) we have 

(7.1) x^ = {-ir E ^^+ E 4'^w^/ 

^S-Podd I:\I\o<m 

\I\a=m 

(7.2) Ym = {-ir J2 ^1+ YI cf\z)Di. 

lePeven I:\I\o<m 

I J|o = "i 

Thus, if we put Z2m-i ■= (-l)"'(-'fm - Ym) for m = 1, . . .,n and Z2m '■= (-l)"'l^m for 
TO = 0, . . . , n — 1 we obtain operators with an expansion 

(7.3) Zp= J2 ^mW^/. 
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where 



(7.4) c(i.)(^)= n^^ n(^^-^^--^) n( 



Zi Zj ^ 



n — i even j — iodd j — i even 



Let A G A be non-zero. Let p be maximal with \p ^ 0. Put fj, := X — (1^) G A. Then we 
have 

(7.5) 5-g^-^Zr G C(iP)(-^a - n)5-g^-x + Y^ k5-g^-„. 

The assumptions on r and a ensure that C(ip)(— ^q. — /u) 7^ 0. The induction hypothesis 
imphes that 5-g^-\ is in the image. D 

7.2. Lemma. Assume r 7^ and a ^ -N - N ■ 2r. Then R\{-Qa) 7^ for all A G A. 

Proof: Suppose R\{—Qct) = 0. For ^u G A let D G 7^ with 6-g^-^ = d-g^D. Since R\ is an 
eigenvector of D we also have R\{—Qct — /u) = 0. This contradicts the fact that —Qa — A 
is Zariski dense in /c"^. D 

Remark: This lemma is only preliminary. Later, we prove the explicit formula (8.9) for 

R\{-Qa)- 

The binomial type theorem, announced in the beginning, is: 

7.3. Theorem. Assume r 7^ and a ^ — N — N ■ 2r. Then for every A G A the following 
formula holds: 

Rx{-a- z) _s^^ 1^|A'|odd.^M(^ + -^) ^m(^) 



(^■6) T7r:; -E(-i) 



R\{-Qa) ^ R^{q + y) R^^-Qa)' 

Proof: The polynomials Rx{—a — z) form also a basis of V"^ . Hence, every / G V"^ has 
an expansion 



(7.7) f{z)=Y,<^^{f)R^{-a-z) 



with a^ G (P^)*. We claim a^ E M with dega^ < |/u|odd- To show this, we evaluate (7.7) 
dX z = —Qa — iJi and get 

(7.8) 5-,„-^(/) = Y. ^r{f)Rr{Q + lA= aM) + E ^r{f)Rr{Q + ^). 

'^ |T|odd<|M|odd 

Then the claim follows by induction on |/u|odd- 

It follows from Proposition 7.1 that there is D^ G TZ with degD^ < |/u|odd such that 
a^(/) = {D^f){-Q^). We apply this to / = Rx- Then 

(7.9) a^{Rx) = {D^Rx){-ga) = P^^io + \)Rx{-Qc?) 
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where p^ '■= Cq^ ^ ^^ ^y Proposition 4.7c). We have degp^ = degD^ < |yu|odd- On the 
other side, we see directly from (7.7) that 

/7ir,N fjD \ Jo if |A|odd < |/u|odd and A 7^ ;u; 

(7.10) a^(i?A) = |(_^)|^u, .^^^^_ 

Thus (7.9), (7.10) together and the very definition of R^{z) imply 

f — DlA^lodd 

(7.11) R^{-Q^)p^{z) = ^^-J-—^R^{z) 

and therefore 

(7.12) „,(«,) = (-l)H-§ik±^Mz^. 

R^{g + fj^) it^(-Pa) 

Now, we insert this into (7.7), replace z by —a — z and obtain (7.6). D 

Remarks: 1. By Theorem 4.9, only those // with // ^ A contribute to the sum in for- 
mula (7.6). In particular, the sum is finite. 

2. The normalizing factor R\{—gct) in the denominator renders the formula more sym- 
metric but causes the restriction on a. Of course, for every a there is an expansion of 
R\{—a — z) in terms of R^{z). It can be easily obtained by using the explicit formula (8.9) 
to calculate the ratio Rx{— Qa) / Rfj.{— Qa) ■ 

There are two immediate applications of the binomial formula (7.6). 

7.4. Corollary. Assume r 7^ and a ^ — N — N ■ 2r. Then the matrix 

'R\{-Qa -^) 



(7.13) , „ , , 

is symmetric. 

Proof: Substitute z = g + u in (7.6). Then the right hand side becomes clearly symmetric 
in A and u. D 



7.5. Corollary. The matrix 

(7.14) ((-i)'^i-^f4^y 

is an involution. 
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Proof: By the binomial formula (7.6), the matrix (7.14) expresses the effect of the involution 
z \-^ —a — z on V^ with respect to the basis „ /_ ^ n , at least if r ^ 0. For r = we argue 
by continuity. D 

The involutory matrix (7.14) can be used to derive an explicit interpolation formula 
(Theorem 7.6iii) below). For this, let C{g + A) be the set of /c- valued functions on ^ + A. 
For f eC{g + A) we define / G C{g + A) by 



(7.15) /(^ + A) := ^(-l)Ho.. ^A.(g + ^) /(g + ^) 



mgA 



Rfj,{g + n)' 



For any fixed A the sum is finite by the Extra Vanishing Theorem 4.9. Let Co{g + A) C 
C{g + A) be the functions with finite support. We consider, via restriction, V^ as a 
subspace of C{g + A). By Lemma 4.4, C{g + A) is a left 7?.-module, provided r 7^ 0. 

7.6. Theorem. The transformation f ^-^ f has the following properties: 

^) / = /• 

tt) feV^^feCoig + A). 

Ill) ForfeV^ holds f{z) = Y.^eA^-l)^^^°-^ f{Q + y)l^^)- 
iv) Assume r 7^ 0. For every D E TZ holds D{f) = Cq f and Cq f = D{f). 



Proof: i) follows from Corollary 7.5 and the fact that the transpose of an involutive matrix 
is involutive. Let Xq+v ^ ^o be the characteristic function of the one-point set {g + i'}- 
Then Xg^i, = {—iy'^^°'^'^Riy{g + h')~^Rjy. Hence, f ^-* f maps a basis of Co to a basis of P^ 
which proves ii). Part Hi) is a direct consequence of i) and ii). 

Finally, let D E TZ. The second formula in iv) follows from the first by i). Thus we 
have to prove 

(7.16) 5{f){g + X) = c^{g + X)f{g + X) 

for every D ElZ, \ E A, and f E C. If we fix D and A then there is a finite subset S C g + A 
such that both sides of (7.16) depend only on values of / in S. Since on 5" every f E C can 
be interpolated by an element of V^ it suffices to prove (7.16) for / = Ri,. But then we 
have 

(7.17) d(r,) = [c^{g + z/)i?,]^ =c^iQ + t^)R^ = c^Ru- 

The last equality holds since Ri^ is a multiple of the characteristic function Xg+i^- D 

7.7. Corollary. Assume r 7^ 0. Let A C Endfc P^ be the algebra generated by V^ and 
IZ. Then there is an involutory automorphism of A which interchanges V^ and IZ. 
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Proof: The automorphism is D h^ D, where D{f) := D{f). Then Theorem 7.6i) imphes 
that this is an involution and part iv) imphes D = Cq for every D eTZ. D 

8. The evaluation formula 

The symmetry of the matrix (7.13) allows to switch the index with the argument. Using 
this, we obtain Fieri type formulas: 

8.1. Theorem. Assume r ^ and a ^ -N -N ■2r. Let D = ^^ c^{z)T^ e 7^. Then for 
all n & K holds 






xeA 

Proof: We substitute z = —Qa — fJ. in the equation Cq{q + i')R^(z) = D{R^){z) and apply 
symmetry (i.e., Corollary 7.4) on both sides. Thus we obtain 

After canceling Ru{—Qoi)j both sides of (8.2) become polynomials in u. Hence we may 
replace u hj —Qa — z. Then putting r] = X — n yields the desired formula. D 

We are applying this to D = X{t) and D = Y{t). By Proposition 4.3, the non-zero 
coefficients are 

c^}'Hz) = {-l)\'\''uf^{z,t)vi{z)wi{z;r), / G Podd 

Cl}'\z) = i-lf^" ur''iz,t)vi{z)wi{z;r), I E Peven 

where 

odd/even 



(8.4) «f'^/^^'^"(^,t) = Hit+z^), vj{z) = n 

i odd /even n — i ev 

(8.5) wi{z-r) = \{{z,-Zj-r)-\{{ 



Zi Zj ^ 



j — i odd j — i even 



After replacing t by a — t we obtain 

JJ(t + zO R^,{z) = 

i odd/even 

(^■^) ( 1 "il^lo T^ ( \ 

E odd/even/ , j.\ / , \ / , \ \~^) ° -'^tJ.[~Qa) j-, / n 

-* '^ -'^odd/even 
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We postpone the simplification of tfiis formula until section 9. Instead, we use (8.6) to 
derive an explicit formula for R\{—gct)- To state the result let 

(8.7) [x t m] := x{x + 1) . . .{x + m — 1) 

be the rising factorial polynomial. For a box s = {i,j) G N^ of a partition A recall the 
following notation: 

a\{s) := Xi — j (arm-length) a'{s) := j — 1 (arm-colength ) 

lx{s) := X'j — i (leg-length) l'{s) := i — 1 (leg-colength ) 

8.2. Theorem. Assume r 7^ 0. Then for every A G A the evaluation formula 

(8.9) Rxi-Qa) = (-l)l^l°<^MA(a)SA, 
holds where 

(8.10) Ax{a):= JJ [a + {n - i)r ^ X,] = JJ {a + a'{s) + {n - l'{s) - l)r) 



l<.i<n sGA 

n — i even n — l'(s) odd 



and 



n [(j-z + l)rrA,-A,] n {a'{s) + {n-l'{s))r) 



l<i<j<n sGA 

^ -1 -1 \ 7-» i — 'i odd n — l'(s) even 

^.11) Sx : = 



n [(j - ^)r T A. - A,] n (axis) + (his) + l)r) 

l<i<j<n s6A 

j — i even ^a(^) odd 

Proof: By continuity, we may assume a ^ — N — N ■ 2r. We expand both sides of (8.6) 
as a polynomial in t and compare coefficients. Then the product on the left-hand side 
becomes ed{zodd/even) while on the right-hand side the wj-factor has to be replaced by 
ed-\i\^{zi\i ^ I odd/even). In particular, only sets / with \I\o < d enter the formula. It 
is easily checked that the set of £7 with / G Podd (resp. / G Peven) with \I\o < d has a 
unique maximum with respect to the order relation [//] < [A], namely (1^) = X]i=i ^i where 
h = 2d — 1 (resp. h = 2d). Thus, by Theorem 6.1, the monomial 2;[^+(^ -•] appears on the 
right-hand side exactly once. Comparing its coefficient, we obtain for all 6 = 1, . . . , n: 

(8.12) R^,+(lb){-gc) = {-iy^^V(^ib){g^ + ij)w(^ib){gc, + iJ,]-r)R^{-gc) 

This is a recursion relation which allows to compute R^{—gct) by deleting one column at 
a time. It follows that R^{—ga) = (— 1)''^'°'^'*^^-B^ with A^j^(^ib-^ = V(^ib^{g + a + fj,)A^ and 
S^+(i!>) = W(^ib){g^ + n; -r)B^. 

The first relation implies easily both formulas for A^. The expressions for B^ could 
be derived in the same way as the analogous formulas [Mac] VI(6.11) and (6.11') for 
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Macdonald polynomials. Especially the second expression for B\ in (8.11) is quite tedious 
to derive which is mainly due to the parity conditions. But there is a trick to derive our 
formulas directly from Macdonald's formulas (6.11) and (6.11'). For any product of the 
form 

(8.13) P = JJ(l-Q"^t^^). 

i 

with Oi, 6i > we define 

(8.14) [P]even:= J] («^ + M- 

{i\bi even} 

The map P i-^ [-Pjeven is multiplicative. The point is now the easily verified formula 
[S^/^jeven = w'(if>)(^a + A*; ^^) with -Bj^/^ as in loc. cit. VI(6.4). From loc. cit. VI(6.10) 
we obtain B^ = [uo{P^)]even- Now the formulas for B^ above are nothing else than [-jeven 
applied to loc. cit. VI(6.11) and (6.11')- □ 

Remarks: 1. The evaluation formula (8.9) are also valid for r = provided one replaces 
the expressions for B\ in (8.2) by their limits for r ^ 0. The same remark holds for all 
the Fieri formulas in section 9. 

2. The polynomials R\{z;r) have, in general, coefficients in Q(r). Conjecturally, one 
obtains an integral form as follows. For A G A let 

(8.15) [cA]even:= J] {ax{s) + {h{s) + l)r) 

sG A 
I \{s) odd 

and Tlx{z;r) := [cA]even ^a(^; r). 

8.3. Conjecture. For all X E A holds TZx{z; r) E Z[r, z]. 

The factor [cA]even is the denominator of the second expression for B\ in (8.2). For n y> 
there is no cancellation involving the variable r. Thus, at least up to a rational factor 
and for big n, the conjectured statement appears to be optimal. The conjecture has been 
tested for n < 6 and |A|odd < 6. (Shifted) Jack polynomials have also certain positivity 
properties (see [KSl] and [KS2]) but none of them seem to generalize to semisymmetric 
polynomials. 

Now, we specialize the evaluation and the binomial formula to the homogeneous poly- 
nomials Rx. The evaluation formula (8.9) becomes 

8.4. Theorem. For X E A holds 



^.16) ^a(1, •••,!) = (^^ if n is odd or [X]i=0; 

I otherwise. 



otherwise. 
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Proof: This follows from (8.9) by calculating lim a ''^^°'^'^Rx{—Qa)- The parity of n comes 



a — »oo 

in because deg^ Ax{a) equals |A|odd (resp. |A|even) if ^ is odd (resp. even). D 



Now, the binomial formula (7.6) becomes: 

8.5. Theorem. Assume r ^ 0. For all X & A holds 

f8 17l ^a(1 + ^) _^ R^{g + X)R^{z) 
^ ' ' Bx ^R^iQ + ^i) B^ 

The sum is over all n E A with /U ^ A and, in case n is even, additionally with [/u]i = [A]i. 

Proof: Using the evaluation formula (8.9), this follows from (7.6) by replacing z by az, 
dividing by the appropriate power a^ and taking the limit a ^ oo. Here 

lAlodd - lAlodd = if 71 is odd; 



t |A|odd - |A|even = [AJi if n IS eveu. Q 

Remark: In the classical case, Lassalle [La] used the expansion of P\{1 + z) to define 
"generalized binomial coefficients". Okounkov-Olshanski [00] proved later the classical 
analogue of (8.17). This implies in particular that Lassalle's binomial coefficients equal 
p'^(, y Thus, classically it is possible to define shifted polynomials from the theory of 
the unshifted ones. The theorem above shows that this fails in the semisymmetric case if 
the number of variables n is even. 

9. The Pieri formula 

Now, we make the Pieri formula completely explicit. Recall again some notation: For 
X = H + ei let Cx/fi (resp. Rx/fi) be the set of boxes of n which are in the same column 
(resp. row) as some box of X\ fx. 

9.1. Theorem. Assume r ^ 0. For every /U G A holds 

(9.1) n (^ + ^') ^'^(^) = E n(^ + ^^ + -"') [^A Jeveo Rx{z) 

i odd/even -^^-^odd/c 



odd/cvcn , , , 

t odd/cvcn 



Here X = n + ej and I runs through -Podd/even • Moreover, 

n {l^r - l^j + (i - ^ - l)r) (A, - A, + {j - i + l)r) 

r I I -1 i<j,j-i odd 

, , ^^V.Jeven- ^ (^, - ^, + (j - z)r) (A. - A, + (j - z)r) 

(9.2) . ^.^.-r.je/ 



z < J, J — t even 



TT Ms) 



sec. R, [^^(^)]^ 
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where 

\ ax{s) + {lx{s) + l)r for lx{s) odd, 

1^ [{ax{s) + 1) + lx{s)r) for lx{s) even. 
Proof: By (8.6) and the evaluation formula (8.9) we have to calculate 



(9.4) [vi{gc,+n)- r^j [wi{ga + fi;-r, 

One easily verifies that the first factor is 1 (not too surprising, given the fact that the result 
can not depend on a). For the second factor we apply the same trick as in the proof of 
Theorem 8.2 and obtain: it is [-Jeven applied to the corresponding formulas for Macdonald 
polynomials. The result follows from [Mac] VI(6.7'), (6.13), and (6.23). D 

Remarks: 1. A priori, it might happen that A = /U + £/ is not a partition. But then 
[V'A/u]even = and the corresponding summand may be omitted. In fact, in that case there 
is an index i such that i^I,j:=i + lEl and Hi = fXi+i. Thus ['(/'wwleven contains the 
factor Hi — Hj + {j — i — l)r = 0. 

2. The first expression for [i^'^, ]even shows that it is a rational function in fj,. The second 
expression takes the cancellation into account which occurs when the "vertical strip" X\n 
contains boxes in the same column, i.e., if there is an index i such that i E I, i + 1 E I and 

By comparing coefficients of powers of t we easily obtain Fieri formulas involving 
elementary symmetric functions: 

9.2. Corollary. 

(9.5) em{z^ I i ,tl) R^{Z) = J] em-s{^i^ + Q^ \i^I,i ^tn) [^V/,]even Rx{z) 

I 

where X = ^ + Sj and where I runs through the elements o/Podd/even with s := \I\o < m. 
Example: For n = 3 we obtain: 

(9.6) (zi+zs) ■ i?^i,^2,^3 = (^i+^3+2r)i?^,,^2,^3 + 

P (^2-^3)(^2-^3-l+2r) 

+/t^,+i,M.,^3 + (^^_^3+2r)(^i-^3-l+2r)^^^''^^'^^+'^ 
, P (^i-^2)(^i-^2-l+2r) 

+«m+i,M2+i,M3 + (^^_^3+2r)(^i-^3-l+2r) ^^'^^+'''^^+' 

(9.7) Z2 ■ -Rmi:A'2,M3 = {|J'2+r)R^^^,^J,2,^^3 + 

, P (^i-^2)(^i-^2-l+2r) p 

+«^i+i,M2+i,M3 + (^^_^3+2r)(;Ui-//3-l+2r) ^^'^^+'''^^+' 
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(9.8) zizs ■ Rfii, H2,^i3 — 



+ (ui+2r) (^2-^3)(^2-^3-l+2r) 



(/Ui-/U3+2r)(;Ui-;U3-l+2r) 



+//3i?^,+i,^2+i,^3 + (^i+2r) 

+-Ra'i + 1,M2 + 1,M3 + 1 



(;»l-;»2)(;»l-;»2-l+2r) 

(/Ui-/U3+2r)(;Ui-;U3-l+2r) 



i? 



+ 1,U3 + 1 + 



A'i,M2i-i,M3 



Formula (9.5) can be used to give a Fieri rule for multiplication with shifted elementary 
semisymmetric polynomials. For this, we introduce the following notation. Let / G Podd, 
s = \I\ and / a semi-symmetric polynomial in n — s variables. Then we define f{z\I') := 
fi^ks+n . . . , Zk^) where i \-^ ki is any parity preserving bijection from {s + 1, . . . , n} to 
{1, . . . , n} \ /. For example, if n = 5, then f{z\{2, 3, 5}') = /(-24, zi). 



9.3. Theorem. For m — 0, . . . ,n and // G A holds 

m 

(9.9) R(^ir^){z)R^{z) =^Y1 R{1^--){Q + 1^\I') Wx/^^even 

s=0 \I\=s 



Rx{z). 



Here X = fx + ej and I runs through -Podd/even according to m odd/even. 

Proof: From the Triangularity Theorem 6.1 (or Corollary 2.8 and the explicit formulas in 
[KSl]) follows that R(^im-j is a linear combination of elementary symmetric functions in the 
odd or even variables. Thus Corollary 9.2 implies that there are semisymmetric functions 

/p(zi, . . . , Zn-p) such that 

(9.10) R^i^-){Z)R^{Z) = J2f\I\iQ + l^\n[^X/^]evenRx{z). 

I 
Here the sum is over all / G -Podd/even if "^ is odd/even. Moreover, (with p = \I\) 

(9.11) deg/p < degP(ir„) - |/|o 



m 
~2 



As one easily checks, the formula 

'm — p 



(9.12) 



y 



holds except when m is even and p is odd, a case which does not occur. Thus we get 
deg/p < degP(im-p) for all m. 

Next, we show that the vanishing conditions hold for fp. For this, let / = {1, . . . ,p}. 
Then /|7|(^ + ;u|/') = fp{gp+i + fXp+i, . . . , gn + ^n)- Put X = ^ + ei. Since ['(/'A/^leven 7^ it 
suffices to show that Pa does not occur in the expansion of P(im)P^ whenever (Up_|_(^_p) = 
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Hm = 0. For this, we put Zm = Qmj ■ ■ ■ ^z^ = Qn- Then the left hand side of (9.10) vanishes 
while, by Proposition 2.5, on the right hand side those -Ra's which don't vanish remain 
linearly independent. Thus, the coefficient in front of i?^_|_£^ is zero which proves the claim. 
We have proved that fp is a multiple of i?(im-p). To show equality we put Zm+i = 
Qm+i, • • • , -Zn = ^n in (9.10) and then replace Zi by Qm + Zi for z = 1, . . . , to. Then i?(im)(z) 
becomes i?(im-)(zi, . . . , Zm) which is the last elementary symmetric polynomial in the odd, 
respectively even, variables. Thus (9.10) becomes simply a special case of Corollary 9.2 
which implies fp = R(^im-py D 

Example: By (2.22) for n = 3 holds R(^i-){z) = zi — Z2 + zs — r. Thus formulas (9.6) and 
(9.7) imply: 

(9.13) i?(i) ■ Rfii,^2,fJ.3 = (^1-^2+^3)-Rmi,M2,A'3 + 

, P (^2-^3)(^2-/^3-l+2r) 

+«Mi+i,M2,M3 + (^^_^3+2r)(^i-^3-l+2r)^^^''^^'^^+' 

in accordance with (9.9), case 171 = 1. Observe also the cancellation which occurs when 
one subtracts (9.7) from (9.6). This is reflected in the fact that in (9.9) for m odd the a 
priori possible terms with \I\ = m + 1 are missing. 

As a consequence of (9.9) we obtain a Fieri rule for the top homogeneous parts: 

9.4. Corollary. For every fx E A and m = 1, . . . , n holds 

(9.14) eUz)Mz) = J2[i^x/>.UenRx{z). 

I 

Here A = fx+ej and I runs through all subsets o/ {1, . . . , n} consisting of [^1 ^dd numbers 
and [y\ even numbers. 

Finally, we complete the explicit computation of Rx, started in Corollary 4.10, where 
A is a hook. 

9.5. Corollary. Let a,m>2 be integers with m even. Then 

a — 1 

(9.15) i?(aim-i) = {R(i) - 1)(-R(i) - 2) . . . {R(i) - a + 2)(i?(i)i?(im) _ i?(im+i)). 

Proof: From formula (9.9) and some short calculations we get 

TTIT 

(9.16) i?(i) ■ i?(„_i 1,^-1) = (a - 2)i?(„_i 1^-1) + R^a 1—1) + (^^_2+^^) (^a-l+mr) ^^""^ ^"^^' 
Thus, 

TTIT 

(9.17) i?(air™-i) = {Rn\-a+2)Rfa-ll"^-^) - 7 TT, r? 7"^ r-R(a-ll'")- 

"■ ' "■ ^ ^ ^ {^a—2+mr){a—l+mr) ^ ' 
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This already implies formula (9.15) for a = 2. For a > 2 we are using induction and 
formula (4.24) for R(^a-ii"^)'- 

a — 2 

-R(al™-i) =(-R(l) - 1) • • • (-R(l) - a + 2)(Rn^Rnm) -— Rnr^+i^)- 

vnv 

(9.18) - 7 TT- V7 TT V^^(i) - 1) ■ ■ ■ (^(1) - a + 2)i?(i™+i) = 

^ ' [a — I ^ rar){a — \ ^ rar) ' ^ ' 

a—\ 

HR{i) -l)(^(i) -2)...(i?(i) -a + 2)(i?(i)i?(i„.) - ^_;^^^^ -^(i'"+^))- 

D 
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